Dimensionality Reduction Algorithms in Machine Learning: A Theoretical and Experimental Comparison†

Abstract

The goal of Feature Extraction Algorithms (FEAs) is to combat the dimensionality curse, which renders machine learning algorithms ineffective. The most representative FEAs are investigated conceptually and experimentally in our work. First, we discuss the theoretical foundation of a variety of FEAs from various categories like supervised vs. unsupervised, linear vs. nonlinear and random-projection-based vs. manifold-based, show their algorithms and compare these methods conceptually. Second, we determine the finest sets of new features for various datasets, as well as in terms of statistical significance, evaluate the eminence of the different types of transformed feature spaces and power analysis, and also determine the FEA efficacy in terms of speed and classification accuracy.