Large margin principles for feature selection

Abstract

In this paper we introduce a margin based feature selection criterion and apply it to measure the quality of sets of features. Using margins we devise novel selection algorithms for multi-class categorization problems and provide theoretical generalization bound. We also study the well known Relief algorithm and show that it resembles a gradient ascent over our margin criterion. We report promising results on various datasets. © Springer-Verlag Berlin Heidelberg 2006.