Emerging Trends in Visual Computing

Abstract

In this paper, we consider the task of clustering multivariate normal distributions with respect to the relative entropy into\r a prescribed number, k, of clusters using a generalization of Lloyd’s k-means algorithm [1]. We revisit this information-theoretic clustering problem under the auspices of mixed-type Bregman divergences,\r and show that the approach of Davis and Dhillon [2] (NIPS*06) can also be derived directly, by applying the Bregman k-means algorithm, once the proper vector/matrix Legendre transformations are defined. We further explain the dualistic structure\r of the sided k-means clustering, and present a novel k-means algorithm for clustering with respect to the symmetrical relative entropy, the J-divergence.Our approach extends to differential entropic clustering of arbitrary members of the same exponential families\r in statistics.