Ensemble Feature Selection Method Based on Recently Developed Nature-Inspired Algorithms

Abstract

Nature-inspired algorithms focus on many real-life applications, majority of which address different types of optimisation problems at a basic level. Feature selection, a type of optimisation problem, is an extremely important aspect of machine learning. This paper proposes an ensemble algorithm for feature selection using four recently developed evolutionary algorithms which are OCSA, OCFA, OBBA and MGWO. A ensemble set is created by combining the reduced feature set(s) obtained from the above-mentioned algorithms. The ensemble set so obtained represents a subset of features that are more robust and stable in nature. Ensemble creates a better composite global model by integrating various “differently biased” classifiers and thus reduces variance error by ensuring diverse “biasing”. The performance of this approach is validated using four classifiers, Decision Tree, Logistic Regression, K-nearest neighbours (KNN) and Random Forest. The application of the proposed method has been demonstrated using ten publicly available datasets. The suggested method shows promising results by either reducing the number of features with not much loss of accuracy or by including more relevant features and thereby increasing accuracy of predictions. Theoretical and empirical results presented in this paper validate the hypothesis that this method can help in finding a better feature subset.