A Study of the Effects of Dimensionality on Stochastic Hill Climbers and Estimation of Distribution Algorithms

Abstract

One of the most important features of an optimization method is its response to an increase in the number of variables, n. Random stochastic hill climber (SHC) and univariate marginal distribution algorithms (UMDA) are two fundamentally different stochastic optimizers. SHC proceeds with local perturbations while UMDA infers and uses a global probability density. The response to dimensionality of the two methods is compared both numerically and theoretically on unimodal functions. SHC response is O(n ln n), while UMDA response ranges from O(√(n) ln(n)) to o(n ln (n)). On two test problems whose sizes go up to 7200, SHC is faster than UMDA. © Springer-Verlag 2004.