PCI: Principal Component Initialization for Deep Autoencoders

Abstract

An autoencoder (AE) is one of the important neural network methods for dimensionality reduction problems. Unfortunately, however, deep AEs have the drawback in trainability which often makes obtaining a good performance a difficult task owing to their model complexity. This paper proposes a simple weight initialization algorithm called the principal component initialization (PCI) method to improve and stabilize the generalization performance of deep AEs in one shot. PCI uses orthogonal bases of the original data space obtained with principal component analysis and transposed ones as initial weights of the AEs. The proposed method significantly outperforms the current de facto standard initialization method for image reconstruction tasks.