Feature deforming for improved similarity-based learning

Abstract

The performance of similarity-based classifiers, such as K-NN, depends highly on the input space representation, both regarding feature relevence and feature interdependence. Feature weighting is a known technique aiming at improving performance by adjusting the importance of each feature at the classification decision. In this paper, we propose a non-linear feature transform for continuous features, which we call feade. The transform is applied prior to classification providing a new set of features, each one resulting by deforming in a local base the original feature according to a generalised mutual information metric for different regions of the feature value range. The algorithm is particularly efficient because it requires linear complexity in respect to the dimensions and the sample and does not need other classifier pre-training. Evaluation on real datasets shows an improvement in the performance of the K-NN classifier.