Studies in Continuous Black-box Optimization

Abstract

Optimization is the research field that studies that studies the design of algorithms for finding the best solutions to problems we humans throw at them. While the whole domain is of important practical utility, the present thesis will focus on the subfield of continuous black-box optimization, presenting a collection of novel, state-of-the-art algorithms for solving problems in that class. First, we introduce a general-purpose algorithm called Natural Evolution Strategies (NES). In contrast to typical evolutionary algorithms which search in the vicinity of the fittest individuals in a population, evolution strategies aim at repeating the type of mutations that led to those individuals. We can characterize those mutations by a search distribution. The key idea of NES is to ascend the gradient on the parameters of that distribution towards higher expected fitness. We show how plain gradient ascent is destined to fail, and provide a viable alternative that instead descends along the natural gradient to adapt the search distribution, which appropriately normalizes the update step with respect to its uncertainty. Being derived from first principles, the NES approach can be extended to all types of search distributions that allow a parametric form, not just the classical multivariate Gaussian one. We derive a number of NES variants for different distributions, and show how they are useful on different problem classes. In addition, we rein in the computational cost, avoiding costly matrix inversions through an incremental change of coordinates. Two additional, novel techniques, importance mixing and adaptation sampling, allow us to automatically tune the learning rate and batch size to the problem, and thereby further reduce the average number of required fitness evaluations. A third technique, restart strategies, provides the algorithm with additional robustness in the presence of multiple local optima, or noise. Second, we introduce a new approach to costly black-box optimization, when fitness evaluations are very expensive. Here, we model the fitness function using state-of-the-art Gaussian process regression, and use the principle of artificial curiosity to direct exploration towards the most informative next evaluation candidate. Both the expected fitness improvement and the expected information gain can be derived explicitly from the Gaussian process model, and our method constructs a front of Pareto-optimal points according to these two criteria. This makes the exploration-exploitation trade-off explicit, and permits maximally informed candidate selection. In summary, this dissertation presents a collection of novel algorithms, for the general problem of continuous black-box optimization as well as a number of special cases, each validated empirically.