What does the evolution path learn in CMA-ES?

Abstract

The Covariance matrix adaptation evolution strategy (CMA-ES) evolves a multivariate Gaussian distribution for continuous optimization. The evolution path, which accumulates historical search directions in successive generations, plays a crucial role in the adaptation of covariance matrix. In this paper, we investigate what the evolution path learns in the optimization procedure. We show that the evolution path accumulates natural gradient with respect to the distribution mean, and acts as a momentum under stationary condition. The experimental results suggest that the evolution path learns relative scales of the eigenvectors, expanded by singular values along corresponding eigenvectors of the inverse Hessian. Further, we show that the outer product of evolution path serves as a rank-1 momentum term for the covariance matrix.