Investigation of distributive data mining calculating architecture

Abstract

A distributive calculating architecture is presented to realize data mining efficiently. The architecture is a hierarchical computational method from the conception, which stores the information in every sub-node with the ideas of database partition, a united center distribute unit can be responsible for the collecting and maintenance of the information in every sub-node, by the scan of database, information is distributed to different nodes, this architecture can maintain a global set enumerate tree, the local large item-sets can be constructed by using any effective algorithm, it mainly solves the problem of highly effective data distribution and data skew, the detailed explaining and theoretical proving of the calculating architecture is given, and how to solve the data skew problem highly and effectively is discussed also in this paper. The partial implementation of this algorithm shows the correctness and feasibility, the calculating architecture can be used for distribute database and most applicable for distribute calculation, which can be used in highly and effectively data mining in distributive and parallel environment.