Utilizing Staging Tables in Data Integration to Load Data into Materialized Views

Abstract

This paper proposes an approach to data integration and migration from a collection of heterogeneous and independent data sources into a data warehouse schema. Current methodology assumes that the data is loaded into data warehouse using queries in order to extract data from multiple data sources. Extracting data from various data sources requires the establishment of generic data integration methodologies. Sometimes, various data anomalies arise when using complex queries across multiple data sources. A data warehouse can be abstractly seen as a set of materialized views. Selecting views for materialization in a data warehouse is one of the most important decisions making tasks in its design. However, there are few facilities in data integration systems that consider these important issues. In this paper, we propose the approach of introducing staging table's schema that can be utilized to load required data from source tables into corresponding staging tables. Staging tables will be assumed to be temporary tables and each staging table will be empty before loading new data and using simple non-join queries to load data. We then focus on an approach to load data into data warehouse materialized views through staging tables including maintenance of materialized views. © Springer-Verlag 2004.