Finding patterns in large star schemas at the right aggregation level

Abstract

There are many stand-alone algorithms to mine different types of patterns in traditional databases. However, to effectively and efficiently mine databases with more complex and large data tables is still a growing challenge in data mining. The nature of data streams makes streaming techniques a promising way to handle large amounts of data, since their main ideas are to avoid multiple scans and optimize memory usage. In this paper we propose in detail an algorithm for finding frequent patterns in large databases following a star schema, based on streaming techniques. It is able to mine traditional star schemas, as well as stars with degenerate dimensions. It is able to aggregate the rows in the fact table that relate to the same business fact, and therefore find patterns at the right business level. Experimental results show that the algorithm is accurate and performs better than the traditional approach. © 2012 Springer-Verlag.