The CMA Evolution Strategy: A Comparing Review

Abstract

Derived from the concept of self-adaptation in evolution strategies, the CMA (Co-variance Matrix Adaptation) adapts the covariance matrix of a multi-variate normal search distribution. The CMA was originally designed to perform well with small populations. In this review, the argument starts out with large population sizes, reflecting recent extensions of the CMA algorithm. Commonalities and differences to continuous Estimation of Distribution Algorithms are analyzed. The aspects of reliability of the estimation, overall step size control, and independence from the coordinate system (invariance) become particularly important in small populations sizes. Consequently, performing the adaptation task with small populations is more intricate. Nomenclature Abbreviations CMA Covariance Matrix Adaptation EDA Estimation of Distribution Algorithm EMNA Estimation of Multivariate Normal Algorithm ES Evolution Strategy (μ/μ {I,W} , λ)-ES, evolution strategy with μ parents, with recombination of all μ parents, either Intermediate or Weighted, and λ offspring. OP : IR n → IR n×n , x → xx T , denotes the outer product of a vector with itself, which is a matrix of rank one with eigenvector x and eigenvalue 2 . RHS Right Hand Side. Greek symbols λ ≥ 2, population size, sample size, number of offspring. μ ≤ λ parent number, number of selected search points in the population. μ cov , parameter for weighting between rank-one and rank-μ update, see (22). μ eff = μ i=1 w 2 i −1 , the variance effective selection mass, see (5). σ (g) ∈ IR + , step size. Latin symbols B ∈ IR n , an orthogonal matrix. Columns of B are eigenvectors of C with unit length and correspond to the diagonal elements of D. C (g) ∈ IR n×n , covariance matrix at generation g. c ii , diagonal elements of C. c c ≤ 1, learning rate for cumulation for the rank-one update of the covariance matrix, see (17) and (33). c cov ≤ 1, learning rate for the covariance matrix update, see (11), (21), (22), and (34). c σ < 1, learning rate for the cumulation for the step size control, see (23) and (31). D ∈ IR n , a diagonal matrix. The diagonal elements of D are square roots of eigen-values of C and correspond to the respective columns of B. d ii , diagonal elements of D. d σ ≈ 1, damping parameter for step size update, see (24), (28), and (32). (0, I), multi-variate normal distribution with zero mean and unity covariance ma-trix. A vector distributed according to N (0, I) has independent, (0, 1)-normally distributed components. N (m, C) ∼ m + N (0, C), multi-variate normal distribution with mean m ∈ IR n