A new and versatile method for association generation

Abstract

Current algorithms for finding associations among the attributes describing data in a database have a number of shortcomings: 1 Applications that require associations with very small support have prohibitively large running times. 2 They assume a static database. Some applications require generating associations in real-time from a dynamic database, where transactions are constantly being added and deleted. There are no existing algorithms to accomodate such applications. 3 They can only find associations of the type where a conjunction of attributes implies a conjunction of different attributes. It turns out that there are many cases where a conjunction of attributes implies another conjunction only provided the exclusion of certain attributes. To our knowledge, there is no current algorithm that can generate such excluding associations. We present a novel method for association generation, that answers all three above desiderata. Our method is inherently different from all existing algorithms, and especially suitable to textual databases with binary attributes. At the heart of our algorithm lies the use of subword trees for quick indexing into the required database statistics. We tested our algorithm on the Reuters-22173 database with satisfactory results.