Information force clustering using directed trees

Abstract

We regard a data pattern as a physical particle experiencing a force acting on it imposed by an overall "potential energy" of the data set, obtained via a non-parametric estimate of Renyi's entropy. The "potential energy" is called the information potential, and the forces are called information forces, due to their information-theoretic origin. We create directed trees by selecting the predecessor of a node (pattern) according to the direction of the information force acting on the pattern. Each directed tree correspond to a cluster, hence enabling us to partition the data set. The clustering metric underlying our method is thus based on entropy, which is a quantity that conveys information about the shape of a probability density, and not only it's variance, as many traditional algorithms based on mere second order statistics rely on. We demonstrate the performance of our clustering technique when applied to both artificially created data and real data, and also discuss some limitations of the proposed method. © Springer-Verlag Berlin Heidelberg 2003.