Scaling Bayesian optimization up to higher dimensions: A review and comparison of recent algorithms

Abstract

Bayesian optimization is known to be a method of choice when it comes to solving optimization problems involving black-box, non-convex and low-dimensional functions in a few iterations. Yet, how to scale this method up to higher dimensions is a challenging and still unsolved research issue. In this paper, we first present and structure recent axes of research addressing this topic. We then experimentally compare three selected high-dimensional Bayesian optimization algorithms to random search on diverse high-dimensional functions. Our results suggest that no algorithm consistently outperforms the others across all types of difficulties encountered and that random search is in general very competitive, confirming recent research results.