Distributed association rule mining with minimum communication overhead

Abstract

In distributed association rule mining algorithm, one of the major and challenging hindrances is to reduce the communication overhead. Data sites are required to exchange lot of information in the data mining process which may generates massive communication overhead. In this paper we propose an association rule mining algorithm which minimizes the communication overhead among the participating data sites. Instead of transmitting all itemsets and their counts, we propose to transmit a binary vector and count of only frequently large itemsets. Message Passing Interface (MPI) technique is exploited to avoid broadcasting among data sites. Performance study shows that the proposed algorithm performs better than two other well known algorithms known as Fast Distributed Algorithm for Mining Association Rules (FDM) and Count Distribution (CD) in terms of communication overhead. © 2009, Australian Computer Society, Inc.