A hierarchical ICA method for unsupervised learning of nonlinear dependencies in natural images

Abstract

Capturing dependencies in images in an unsupervised manner is important for many image-processing applications and understanding the structure of natural image signals. Linear generative models such as independent component analysis (ICA) have shown to capture low level features such as oriented edges in images. However ICA only captures linear dependency due to its linear model constraints and its modeling capability is limited. We propose a new method for capturing nonlinear dependencies in natural images. It is an extension of the linear ICA method and builds on a hierarchical representation. It makes use of lower level linear ICA representation and a subsequent mixture of Laplacian distribution for learning the nonlinear dependencies. The model is learned via the EM algorithm and it can capture variance correlation and high order structures in a consistent manner. We visualize the learned variance structure and demonstrate applications to image segmentation and denoising. © Springer-Verlag 004.