Parallelizing em clustering algorithm on a cluster of SMPs

Abstract

In this paper, we report on parallelization of the EM clustering algorithm using the FREERIDE middleware developed in our prior work. FREERIDE is based upon the observation that the processing structure of a large number of data mining algorithms involves generalized reductions. FREERIDE offers a high-level interface and support both distributed memory and shared memory parallelization, besides efficient execution on disk-resident datasets. We show how the main processing loops in both the E and M steps of the EM algorithm essentially involve a generalized reduction, and therefore, the algorithm can be parallelized using FREERIDE. © Springer-Verlag 2004.