Using Incremental Pruning to Increase the Efficiency of Dynamic Itemset Counting for Mining Association Rules

Abstract

Dynamic item set counting (DIC) is an efficient method to search for frequent sets in mining association rules in large database systems. Compared with many other optimization techniques, it not only has the potential to reduce the number of database scanning, but also never generates more candidate sets than that generated in Apriori. However, the original proposal of DIC employs only the mechanism to prune the candidate sets that is used in Apriori algorithm. This can still result in unnecessary scanning in certain cases. In this paper, we propose a scheme to minimizing the unnecessary scanning by using an additional pruning method on top of the existing one. This pruning method can progressively eliminate some candidate sets that are not frequent. We present experimental results that show by using this mechanism the performance can indeed be improved.