The sparse coding method is formulated as an information theoretic optimization problem. The rate distortion theory leads to an objective functional which can be interpreted as an information theoretic formulation of the sparse coding. Viewing as an entropy minimization problem, the rate distortion theory and consequently the sparse coding are extended to discriminative variants. As a concrete example of this information theoretic sparse coding, a discriminative non-linear sparse coding algorithm with neural networks is proposed. Experimental results of gender classification by face images show that the discriminative sparse coding is more robust to noise, compared to the conventional method which directly uses images as inputs to a linear support vector machine. © 2009 Springer Berlin Heidelberg.
CITATION STYLE
Hino, H., & Murata, N. (2009). An information theoretic perspective of the sparse coding. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Vol. 5551 LNCS, pp. 84–93). https://doi.org/10.1007/978-3-642-01507-6_11
Mendeley helps you to discover research relevant for your work.