Hybrid Feature Selection Method Based on Feature Subset and Factor Analysis

Abstract

With the advent of big data era and the rapid improvement of raw data scale, feature selection, as the basis and critical technologies for data mining, plays an increasingly important role. However, most studies on feature selection methods, mainly directed to treat the single feature or overall feature subset, while the influence of the correlation and redundancy of features in the feature subset on the classification results is ignored. In this paper, a hybrid feature selection method based on feature subsets generated by factor analysis (FAFS_HFS) is proposed. Firstly, this method generates feature subsets from the maximum load (maximum explanatory power) of each feature through factor analysis. Then, minimal redundancy and maximal relevance (mRMR) and sequential forward selection (SFS) are used to remove the redundancy of each feature subset. Finally, fisher score based on feature subset (FSF-score) is utilized to evaluate and obtain the optimal feature subsets. Experiments are conducted on 14 datasets, the results show that FAFS_HFS method has higher classification accuracy and lower dimension on almost all datasets, especially in high-dimensional datasets, and it has competitive efficiency and classification performance compared with other methods.