Geometric Methods for Feature Extraction and Dimensional Reduction - A Guided Tour

Abstract

We give a tutorial overview of several geometric methods for fea-ture selection and dimensional reduction. We divide the methods into projective methods and methods that model the manifold on which the data lies. For projective methods, we review projection pursuit, principal component analysis (PCA), kernel PCA, probabilistic PCA, and oriented PCA; and for the manifold methods, we review multidi-mensional scaling (MDS), landmark MDS, Isomap, locally linear em-bedding, Laplacian eigenmaps and spectral clustering. The Nyström method, which links several of the algorithms, is also reviewed. The goal is to provide a self-contained review of the concepts and mathe-matics underlying these algorithms.