Using kNN model for automatic feature selection

Abstract

This paper proposes a kNN model-based feature selection method aimed at improving the efficiency and effectiveness of the ReliefF method by: (1) using a kNN model as the starter selection, aimed at choosing a set of more meaningful representatives to replace the original data for feature selection; (2) integration of the Heterogeneous Value Difference Metric to handle heterogeneous applications - those with both ordinal and nominal features; and (3) presenting a simple method of difference function calculation based on inductive information in each representative obtained by kNN model. We have evaluated the performance of the proposed kNN model-based feature selection method on toxicity dataset Phenols with two different endpoints. Experimental results indicate that the proposed feature selection method has a significant improvement in the classification accuracy for the trial dataset. © Springer-Verlag Berlin Heidelberg 2005.