On the equivalence of top-down and bottom-up data mining in relational databases

Abstract

Although knowledge discovery from large relational databases has gained popularity, and its significance is well recognized, the prohibitive nature of the cost associated with extracting such knowledge, and the lack of suitable declarative query language support, still act as limiting factors. Surprisingly, little or no relational technology has yet been significantly exploited in data mining even though data often reside in relational tables. Consequently, no relational optimization has yet been possible for data mining. We exploit the transitive nature of large item sets and the so called anti-monotonicity property of support thresholds of large item sets to develop a natural least fixpoint operator for data mining. The operator proposed has several advantages including optimization opportunities, and traditional candidate set free large item set generation. We present an SQL3 expression for association rule mining and discuss its mapping to the least fixpoint operator developed in this paper, and thereby establish the equivalence of the top-down and bottom-up computation of large item sets in relational databases.