Partition selection approach for hierarchical clustering based on clustering ensemble

Abstract

Hierarchical clustering algorithms are widely used in many fields of investigation. They provide a hierarchy of partitions of the same dataset. However, in many practical problems, the selection of a representative level (partition) in the hierarchy is needed. The classical approach to do so is by using a cluster validity index to select the best partition according to the criterion imposed by this index. In this paper, we present a new approach based on the clustering ensemble philosophy. The representative level is defined here as the consensus partition in the hierarchy. In the consensus computation process, we take into account the similarity between partitions and information from the evaluation of partitions with different cluster validity indexes. An experimental comparison on several datasets shows the superiority of the proposed approach with respect to the classical approach. © 2010 Springer-Verlag.