Sparse kernel-based feature weighting

Abstract

The success of many learning algorithms hinges on the reliable selection or construction of a set of highly predictive features. Kernel-based feature weighting bridges the gap between feature extraction and subset selection. This paper presents a rigorous derivation of the Kernel-Relief algorithm and assesses its effectiveness in comparison with other state-of-art techniques. For practical considerations, an online sparsification procedure is incorporated into the basis construction process by assuming that the kernel bases form a causal series. The proposed sparse Kernel-Relief algorithm not only produces nonlinear features with extremely sparse kernel expressions but also reduces the computational complexity significantly. © 2008 Springer-Verlag Berlin Heidelberg.