Softening some effects of the no free lunch (NFL) theorems in optimization via parallel computing

Abstract

The no free lunch (NFL) theorems, aptly named by Wolpert and Macready, imply that no search or optimization (SO) algorithm will always outperform all other SO algorithms. Not only may an algorithm fare very different when applied to different classes of problems, but different implementations of a given algorithmic paradigm may result in the same. In this paper, parallel computing is used to soften this effect of the NFL theorem which is very well known in practice. In this parallel paradigm different optimization solvers compete in parallel to solve a nonlinear optimization subproblem. The first successful iterate is then accepted. This approach allows us to solve SO problems far more efficiently than is possible with a single solver. Our numerical experiments show that the most suitable solver, usually selected a priori, may change as the iterations progress, contrary to intuition. © 2013 Taylor & Francis Group.