A Comparison of Feature Construction Methods in the Context of Supervised Feature Election for Classification

Abstract

In supervised machine learning applications, feature construction may be used to create additional, informative features with the aim to support the prediction of the target output. This study investigates the impact of feature construction, specifically the use of quadratic and interaction terms, on the predictive performance of a classifier. Moreover, the Yager intersection operator is applied as a feature construction method to form additional interaction features. Since feature construction may also create irrelevant features, it can be combined with feature selection to maintain or even reduce the dimensionality of the feature set for model training. In this study, the supervised feature selection method ReliefF is used to rank all features and the k-nearest neighbor classifier is used for predicting the target classes. On the seven real-world data sets contained in this study, the features generated using feature construction are often among the most important features and provide competitive predictive performance.