Mining correlated patterns with multiple minimum all-confidence thresholds

Abstract

Correlated patterns are an important class of regularities that exist in a database. The all -con f idence measure has been widely used to discover the patterns in real-world applications. This paper theoretically analyzes the all-confidence measure, and shows that, although the measure satisfies the null-invariant property, mining correlated patterns involving both frequent and rare items with a single minimum all-confidence (minAllCon f ) threshold value causes the "rare item problem" if the items' frequencies in a database vary widely. The problem involves either finding very short length correlated patterns involving rare items at a high minAllCon f threshold, or generating a huge number of patterns at a low minAllCon f threshold. The cause for the problem is that the single minAllCon f threshold was not sufficient to capture the items' frequencies in a database effectively. The paper also introduces an alternative model of correlated patterns using the concept of multiple minAllCon f thresholds. The proposed model facilitates the user to specify a different minAllCon f threshold for each pattern to reflect the varied frequencies of items within it. Experiment results show that the proposed model is very effective. © Springer-Verlag 2013.