Differences between clusterings as distances in the covering graph of partitions

Abstract

Cluster analysis in data mining often requires to quantify the difference between alternative clusterings or partitions of a fixed data set: typically, when varying the input in multiple runs of a local-search clustering algorithm, it is important to measure how apart the resulting outputs actually are. This work addresses the issue by means of paths in the covering graph of partitions, hence dealing with these latter as lattice elements. After paralleling the standard Hamming distance between subsets by counting how many atoms of the lattice are included in a symmetric difference between partitions, the approach is generalized by weighting the edges in the covering graph of the lattice through arbitrary symmetric, order-preserving/inverting and super/submodular partition functions, examples being rank and entropy. The induced metrics then obtain as the overall weight of lightest paths in such a weighted graph.