Continuous Hyper-parameter Configuration for Particle Swarm Optimization via Auto-tuning

Abstract

Hyper-Parameter configuration is a relatively novel field of paramount importance in machine learning and optimization. Hyper-parameters refers to the parameters that control the behavior of algorithms and are not tuned directly by such algorithms. For hyper-parameters of an optimization algorithm such as Particle Swarm Optimization, hyper-parameter configuration is a nested optimization problem. Usually, practitioners needs to use a second optimization algorithm such as grid search or random search to find proper hyper-parameters. However, this approach forces practitioners to know about two different algorithms. Moreover, hyper-parameter configuration algorithms also have hyper-parameters that need to be considered. In this work we use Particle Swarm Optimization to configure its own hyper-parameters. Results show that hyper-parameters configured by PSO are competitive with hyper-parameters found by other hyper-parameter configuration algorithms.