Phase transitions in soft topographic vector quantization

Abstract

We have developed an algorithm (STVQ) for the optimization of neighborhood preserving maps by applying deterministic annealing to an energy function for topographic vector quantization. The combinatorial optimization problem is solved by introducing temperature dependent fuzzy assignments of data points to cluster centers and applying an EM-type algorithm at each temperature while annealing. The annealing process exhibits phase transitions in the cluster representation for which we calculate critical modes and temperatures expressed in terms of the neighborhood function and the covariance matrix of the data. In particular, phase transitions corresponding to the automatic selection of feature dimensions are explored analytically and numerically for finite temperatures. Results are related to those obtained earlier for Kohonen's SOM-algorithm which can be derived as an approximation to STVQ. The deterministic annealing approach makes it possible to use the neighborhood function solely to encode desired neighborhood relations. The working of the annealing process is visualized by showing the effects of "heating" on the topological structure of a two-dimensional map of the plane.