ESOM visualizations for quality assessment in clustering

Abstract

Classical clustering algorithms as well as intrinsic evaluation criteria impose predefined structures onto a data set. If the structures do not fit the data, the clustering will fail and the evaluation criteria will lead to erroneous conclusions. Recently, the abstract U-matrix has been defined for emergent self-organizing maps (ESOM). In this work the abstract forms of the P- and the U* are defined in anal- ogy to the P- and the U*-matrix on ESOM. The abstract U*-matrix can be used for AU*-clustering of data by taking account of density and distance structures. For AU*- clustering the structures seen on the ESOM serve as a supervising quality measure. In this way it can be determined whether an AU*-clustering represents important structures inherent to the high dimensional data. Importantly, AU*-clustering does not impose a geometric cluster shape, which may not fit the underlying data structure, onto the data set. The approach is demonstrated on benchmark data as well as real world data from spatial science.