Data mining on desktop grid platforms

Abstract

Very large data volumes and high computation costs in data mining applications justify the use for them of Grid-level massive parallelism. The paper concerns Grid-oriented implementation of the DisDaMin (Distributed Data Mining) project, which proposes distributed knowledge discovery through parallelization of data mining tasks. DisDaMin solves data mining problems by using new distributed algorithms based on special clusterized data decomposition and asynchronous task processing, which match the Grid computing features. The DisDaMin algorithms are embedded inside the DG-ADAJ (Desktop-Grid Adaptative Application in Java) system, which is a middleware platform for Desktop Grid. It provides adaptive control of distributed applications written in Java for Grid or Desktop Grid. It allows an optimized distribution of applications on clusters of Java Virtual Machines, monitoring of application execution and dynamic on-line balancing of processing and communication. Simulations were performed to prove the efficiency of the proposed mechanisms. They were carried on using the French national project Grid'5000 (part of the CoreGrid project) and the DG-ADAJ. © 2008 Springer-Verlag Berlin Heidelberg.