mRMR+: An Effective Feature Selection Algorithm for Classification

Abstract

This paper presents an empirical study using three entropy measures such as Shannon’s entropy, Renyi’s entropy, and Tsallis entropy, while calculating mutual information to select top ranked features. We evaluate the selected features using three established classifiers such as naive Bayes, IBK and Random Forest in terms of classification accuracy on five gene expression datasets. We observe that none gives consistent performance in ordering the features based on their rank. To address this issue, we propose a variant of mRMR, using ensemble approach based on our own weight function. The results establish that our method is significantly superior than its other counterparts in terms of feature selection and classification accuracy in most of the datasets.