Towards the development of large-scale data warehouse application frameworks

Abstract

Facing with growing data volumes and deeper analysis requirements, current development of Business Intelligence (BI) and Data warehousing systems (DWHs) is a challenging and complicated task, which largely involves in ad-hoc integration and data re-engineering. This arises an increasing requirement for a scalable application framework which can be used for the implementation and administration of diverse BI applications in a straight forward and cost-efficient way. In this context, this paper presents a large-scale application framework for standardized BI applications, supporting the ability to define and construct data warehouse processes, new data analytics capabilities as well as to support the deployment requirements of multi scalable front-end applications. The core of the framework consists of defined metadata repositories with pre-built and function specific information templates as well as application definition. Moreover, the application framework is also based on workflow mechanisms for developing and running automatic data processing tasks. Hence, the framework is capable of offering an unified reference architecture to end users, which spans various aspects of development lifecycle and can be adapted or extended to better meet application-specific BI engineering process. © 2012 IFIP International Federation for Information Processing.