Benchmarking and evaluating MATLAB derivative-free optimisers for single-objective applications

Abstract

MATLAB® builds in a number of derivative-free optimisers (DFOs), conveniently providing tools beyond conventional optimisation means. However, with the increase of available DFOs and being compounded by the fact that DFOs are often problem dependent and parameter sensitive, it has become challenging to determine which one would be most suited to the application at hand, but there exist no comparisons on MATLAB DFOs so far. In order to help engineers use MATLAB for their applications without needing to learn DFOs in detail, this paper evaluates the performance of all seven DFOs in MATLAB and sets out an amalgamated benchmark of multiple benchmarks. The DFOs include four heuristic algorithms - simulated annealing, particle swarm optimization (PSO), the genetic algorithm (GA), and the genetic algorithm with elitism (GAe), and three direct-search algorithms - Nelder-Mead’s simplex search, pattern search (PS) and Powell’s conjugate search. The five benchmarks presented in this paper exceed those that have been reported in the literature. Four benchmark problems widely adopted in assessing evolutionary algorithms are employed. Under MATLAB’s default settings, it is found that the numerical optimisers Powell is the aggregative best on the unimodal Quadratic Problem, PSO on the lower dimensional Scaffer Problem, PS on the lower dimensional Composition Problem, while the extra-numerical genotype GAe is the best on the Varying Landscape Problem and on the other two higher dimensional problems. Overall, the GAe offers the highest performance, followed by PSO and Powell. The amalgamated benchmark quantifies the advantage and robustness of heuristic and population-based optimisers (GAe and PSO), especially on multimodal problems.