A parallel and distributed method for computing high dimensional MOLAP

Abstract

Data cube has been playing an essential role in fast OLAP(on-line analytical processing) in many multidimensional data warehouse. We often execute range queries on aggregate cube computed by pre-aggregate technique in MOLAP. For the cube with d dimensions, it can generate 2 cuboids. But in a high-dimensional data warehouse (such as the applications of bioinformatics and statistical analysis, etc.), we build all these cuboids and their indices and full materialized the data cube impossibly. In this paper, we propose a multi-dimensional hierarchical fragmentation of the fact table based on dimension hierarchical encoding. This method partition the high dimensional data cube into shell mini-cubes. Using dimension hierarchical encoding and pre-aggregated results, OLAP queries are computed online by dynamically constructing cuboids from the fragment data cubes. Such an approach permits a significant reduction of processing and I/O overhead for many queries by restricting the number of fragments to be processed for both the fact table and bitmap encoding data. This method also supports parallel I/O and parallel processing as well as load balancing for disks and processors. We have compared the methods of our parallel method with the other existed ones such as partial cube by experiment. The analytical and experimental results show that the method of our parallel method proposed in this paper is more efficient than the other existed ones. © IFIP International Federation for Information Processing 2005.