PCA-enhanced stochastic optimization methods

Abstract

In this paper, we propose to enhance particle-based stochastic optimization methods (SO) by using Principal Component Analysis (PCA) to build an approximation of the cost function in a neighborhood of particles during optimization. Then we use it to shift the samples in the direction of maximum cost change. We provide theoretical basis and experimental results showing that such enhancement improves the performance of existing SO methods significantly. In particular, we demonstrate the usefulness of our method when combined with standard Random Sampling, Simulated Annealing and Particle Filter. © 2012 Springer-Verlag.