Energy Correlation of Linear Features and Normalization

Abstract

It turns out that when we estimate ICA from natural images, the obtained components are not really independent. This may be surprising since after all, in the ICA model, the components are assumed to be independent. But it is important to understand that while the components in the theoretical model are independent, the estimates of the components of real image data are often not independent. What ICA does is that it finds the most independent components that are possible by a linear transformation, but a linear transformation has so few parameters that the estimated components are often quite far from being independent. In this chapter and the following ones, we shall consider some dependencies that can be observed between the estimated independent components. They turn out to be extremely interesting both from the viewpoint of computational neuroscience and image processing. Like in the case of ICA, the models proposed here are still very far from providing a complete description of natural image statistics, but each model does exhibit some very interesting new phenomena just like ICA.