In this chapter, we discuss important connections between two different approaches to machine learning, namely Renyi entropy-based information theoretic learning and the Mercer kernel methods. We show that Parzen windowing for estimation of probability density functions reveals the connections, enabling the information theoretic criteria to be expressed in terms of mean vectors in a Mercer kernel feature space, or equivalently, in terms of kernel matrices. From this we learn not only that two until now separate paradigms in machine learning are related, it also enables us to interpret and understand methods developed in one paradigm in terms of the other, and to develop new sophisticated machine learning algorithms based on both approaches.
Mendeley saves you time finding and organizing research
Choose a citation style from the tabs below