Kernel principal components are maximum entropy projections

Abstract

Principal Component Analysis (PCA) is a very well known statistical tool. KERNEL PCA is a nonlinear extension to PCA based on the kernel paradigm. In this paper we characterize the projections found by KERNEL PCA from a information theoretic perspective. We prove that KERNEL PCA provides optimum entropy projections in the input space when the Gaussian kernel is used for the mapping and a sample estimate of Renyi's entropy based on the Parzen window method is employed. The information theoretic interpretation motivates the choice and specifices the kernel used for the transformation to feature space. © Springer-Verlag Berlin Heidelberg 2006.