Summarizing cluster evolution in dynamic environments

Abstract

Monitoring and interpretation of changing patterns is a task of paramount importance for data mining applications in dynamic environments. While there is much research in adapting patterns in the presence of drift or shift, there is less research on how to maintain an overview of pattern changes over time. A major challenge lays in summarizing changes in an effective way, so that the nature of change can be understood by the user, while the demand on resources remains low. To this end, we propose FINGERPRINT, an environment for the summarization of cluster evolution. Cluster changes are captured into an "evolution graph", which is then summarized based on cluster similarity into a fingerprint of evolution by merging similar clusters. We propose a batch summarization method that traverses and summarizes the Evolution Graph as a whole, and an incremental method that is applied during the process of cluster transition discovery. We present experiments on different data streams and discuss the space reduction and information preservation achieved by the two methods. © 2011 Springer-Verlag.