An annealed 'neural gas' network for robust vector quantization

Abstract

Vector quantization, a central topic in data compression, deals with the problem of encoding an information source or a sample of data vectors by means of a finite codebook, such that the average distortion is minimized. We introduce a common framework, based on maximum entropy inference to derive a deterministic annealing algorithm for robust vector quantization. The objective function for codebook design is extended to take channel noise and bandwidth limitations into account. Formulated as an on-line problem it is possible to derive learning rules for competitive neural networks. The resulting update rule is a generalization of the 'neural gas' model. The foundation in coding theory allows us to specify an optimality criterion for the 'neural gas' update rule.