Ensemble features selection algorithm by considering features ranking priority

Abstract

Feature selection is a pre-processing for choosing relevant features and ignores features that tend to have no predictive information. Feature selection is applied to improve the accuracy of classification process. High relevant features have a tendency to get high classification performance. This paper proposed the ensemble of multiple feature ranking techniques by considering ranker priority for feature selection. Five individual feature ranking algorithms (information gain, gain ratio, symmetrical uncertainty, reliefF and oneR) are investigated and considered together as ensemble, based on ranking priority. The lung cancer, lymphoma, breast cancer, ovarian cancer and leukemia datasets were gathered from Kent Ridge bio-medical data and Machine Learning data repository. The datasets are applied to ensemble features selection algorithm. The obtained results are compared to results from individual feature ranking algorithms and the existing ensemble algorithm. The selected features are applied to classification algorithms. Area under the curve (AUC), precision and recall values from six classification algorithms are used to evaluate the obtained features. The experimental results show that the selected features from proposed ensemble features selection algorithm are greater than those of individual feature ranking techniques and the existing ensemble features selection algorithm.