Mining non-coincidental rules without a user defined support threshold

Abstract

Traditional association rule mining techniques employ the support and confidence framework. However, specifying minimum support of the mined rules in advance often leads to either too many or too few rules, which negatively impacts the performance of the overall system. Here we propose replacing Apriori's user-defined minimum support threshold with the more meaningful MinAbsSup function. This calculates a custom minimum support for each itemset based on the probability of chance collision of its items, as derived from the inverse of Fisher's exact test. We will introduce the notion of coincidental itemsets; given a transaction dataset there is a chance that two independent items are appearing together by random coincidence. Rules generated from these itemsets do not denote a meaningful association, and are not useful. © 2008 Springer-Verlag Berlin Heidelberg.