Comparison of redundancy and relevance measures for feature selection in tissue classification of CT images

Abstract

In this paper we report on a study on feature selection within the minimum-redundancy maximum-relevance framework. Features are ranked by their correlations to the target vector. These relevance scores are then integrated with correlations between features in order to obtain a set of relevant and least-redundant features. Applied measures of correlation or distributional similarity for redunancy and relevance include Kolmogorov-Smirnov (KS) test, Spearman correlations, Jensen-Shannon divergence, and the sign-test. We introduce a metric called "value difference metric" (VDM) and present a simple measure, which we call "fit criterion" (FC). We draw conclusions about the usefulness of different measures. While KS-test and sign-test provided useful information, Spearman correlations are not fit for comparison of data of different measurement intervals. VDM was very good in our experiments as both redundancy and relevance measure. Jensen-Shannon and the sign-test are good redundancy measure alternatives and FC is a good relevance measure alternative. © 2010 Springer-Verlag Berlin Heidelberg.