Change discovery in heterogeneous data sources of a data warehouse

Abstract

Data warehouses have been used to analyze data stored in relational databases for several decades. However, over time, data that are employed in the decision-making process have become so enormous and heterogeneous that traditional data warehousing solutions have become unusable. Therefore, new big data technologies have emerged to deal with large volumes of data. The problem of structural evolution of integrated heterogeneous data sources has become extremely topical due to dynamic and diverse nature of big data. In this paper, we propose an approach to change discovery in data sources of a data warehouse utilized to analyze big data. Our solution incorporates an architecture that allows to perform OLAP operations and other kinds of analysis on integrated big data and is able to detect changes in schemata and other characteristics of structured, semi-structured and unstructured data sources. We discuss the algorithm for change discovery and metadata necessary for its operation.