Determination of the data model for heterogeneous data processing based on cost estimation

Abstract

In heterogeneous data processing, various data model often make analytic task too hard to achieve optimal performance, it is necessary to unify heterogeneous data into the same data model. How to determine the proper intermediate data model and unify the involved heterogeneous data models for the analytical task is an urgent problem need to be solved. In this paper, we proposed a model determination method based on cost estimation. It evaluates the execution cost of query tasks on different data models, which taken as the criterion to measure the data model, and chooses a data model with the least cost as the intermediate representation during data processing. The experimental results of BigBench datasets showed that the proposed cost estimation based method could appropriately determine the data model, which made heterogeneous data processing efficiently.