StreamLeader: A new stream clustering algorithm not based in conventional clustering

Abstract

Stream clustering algorithms normally require two phases: an online first step that statistically summarizes the stream while forming special structures – such as micro-clusters– and a second, offline phase, that uses a conventional clustering algorithm taking the microclusters as pseudo-points to deliver the final clustering. This procedure tends to produce oversized or overlapping clusters in medium-to-high dimensional spaces, and typically degrades seriously in noisy data environments. In this paper we introduce StreamLeader, a novel stream clustering algorithm suitable to massive data that does not resort to a conventional clustering phase, being based on the notion of Leader Cluster and on an aggressive noise reduction process. We report an extensive systematic testing in which the new algorithm is shown to consistently outperform its contenders both in terms of quality and scalability.