Independent variable group analysis

Abstract

When modeling large problems with limited representational resources, it is important to be able to construct compact models of the data. Structuring the problem into sub-problems that can be modeled independently is a means for achieving compactness. In this article we introduce Independent Variable Group Analysis (IVGA), a practical, efficient, and general approach for obtaining sparse codes. We apply the IVGA approach for a situation where the dependences within variable groups are modeled using vector quantization. In particular, we derive a cost function needed for model optimization with VQ. Experimental results are presented to show that variables are grouped according to statistical independence, and that a more compact model ensues due to the algorithm.