An improved FP-growth algorithm based on SOM partition

Abstract

FP-growth algorithm is an algorithm for mining association rules without generating candidate sets. It has high practical value in many fields. However, it is a memory resident algorithm, and can only handle small data sets. It seems powerless when dealing with massive data sets. This paper improves the FP-growth algorithm. The core idea of the improved algorithm is to partition massive data set into small data sets, which would be dealt with separately. Firstly, systematic sampling methods are used to extract representative samples from large data sets, and these samples are used to make SOM (Self-organizing Map) cluster analysis. Then, the large data set is partitioned into several subsets according to the cluster results. Lastly, FP-growth algorithm is executed in each subset, and association rules are mined. The experimental result shows that the improved algorithm reduces the memory consumption, and shortens the time of data mining. The processing capacity and efficiency of massive data is enhanced by the improved algorithm.