An overcomplete ICA algorithm by InfoMax and InfoMin

Abstract

It is known that the number of the edge detectors significantly exceeds that of input signals in the visual system of the brains. This phenomenon has been often regarded as overcomplete independent component analysis (ICA) and some generative models have been proposed. Though the models are effective, they need to assume some ad-hoc prior probabilistic models. Recently, the InfoMin principle was proposed as a comprehensive framework with minimal prior assumptions for explaining the information processing in the brains and its usefulness has been verified in the classic non-overcomplete cases. In this paper, we propose a new ICA contrast function for overcomplete cases, which is deductively derived from the the InfoMin and InfoMax principles without any prior models. Besides, we construct an efficient fixed-point algorithm for optimizing it by an approximate Newton's method. Numerical experiments verify the effectiveness of the proposed method. © Springer-Verlag Berlin Heidelberg 2008.