Data Warehouse Modernization Using Document-Oriented ETL Framework for Real Time Analytics

Abstract

For past several years Relational Databases performing and providing services to many applications. As data is growing, the necessity for a new kind of database which can handle such a huge amount of data in semistructured and unstructured form has also increased. NoSQL databases are able to manage complexity of data structure as well as they can handle such a huge amount of data. Analysis is again a challenging process of such semi-structured and unstructured data. Traditional data warehouses are incapable of analyzing such schema-less data for decision making as relational databases need to know schema in advance. ETL (Extract-Transform-Load) is the process used to collect data from source, process and transform them and storing into a data warehouse for further analysis. The paper presents a framework for an ETL process for document-oriented data warehouse which provides real-time analytics using classification approach at the data warehouse stage. The proposed framework is designed and verified to enhance execution time for real-time analytics.