Finding rare patterns with weak correlation constraint: Progress in indicative and chance patterns

Abstract

A notion of rare patterns has been recently paid attention in several research fields including Chance Discovery, Formal Concept Analysis and Data Mining. In this paper, we overview the progress of our investigations on rare patterns satisfying a weak-correlation constraint. A rare pattern must indicate some significance as well as a fact that the number of its instances is a few. We pay our attention to a pattern as an itemset in a transaction database which consists of several general items, but has a very small degree of correlation in spite of the generality of component items. Such a pattern is called an indicative pattern and is regarded as a rare pattern to be extracted. In order to exclude trivial patterns of general items with few instances, we introduce an objective function for taking into account both the generality of component items and the number of instances as objective evidences. Then we try to find indicative patterns with the Top-N evaluation values under a constraint that the degree of correlation must not exceed a given upper bound. For making a hidden relationship between a pair of more frequent patterns visible, the framework of finding Top-N indicative patterns is then extended by imposing some structural constraints to our indicative pattern and larger patterns bridged by it. As a recent progress in this direction, we briefly present a framework of finding chance patterns with KeyGraph-based importance as well as some experimental result.