A tutorial on principal component analysis: derivation, discussion and singular value decomposition

Abstract

Principal component analysis (PCA) is a mainstay of modern data analysis - a black box that is widely used but poorly understood. The goal of this paper is to dispel the magic behind this black box. This tutorial focuses on building a solid intuition for how and why principal component analysis works; furthermore, it crystallizes this knowledge by deriving from first prin-cipals, the mathematics behind PCA . This tutorial does not shy away from explaining the ideas infor-mally, nor does it shy away from the mathematics. The hope is that by addressing both aspects, readers of all levels will be able to gain a better understand-ing of the power of PCA as well as the when, the how and the why of applying this technique.