What Makes Particle Swarm Optimization a Very Interesting and Powerful Algorithm?

Abstract

Particle swarm optimization (PSO) is an evolutionary computational technique used for optimization motivated by the social behavior of individuals in large groups in nature. Different approaches have been used to understand how this algorithm works and trying to improve its convergence properties for different kind of problems. These approaches go from heuristic to mathematical analysis, passing through numerical experimentation. Although the scientific community has been able to solve a big variety of engineering problems, the tuning of the PSO parameters still remains one of its major drawbacks. This chapter reviews the methodology developed within our research group over the last three years, which is based in adopting a completely different approach than those followed by most of the researchers in this field. By trying to avoid heuristics we proved that PSO can be physically interpreted as a particular discretization of a stochastic damped mass-spring system. Knowledge of this analogy has been crucial in deriving the PSO continuous model and to deduce a family of PSO members with different properties with regard to their exploitation/exploration balance: the generalized PSO (GPSO), the CC-PSO (centered PSO), CP-PSO (centered-progressive PSO), PP-PSO (progressive-progressive PSO) and RR-PSO (regressive-regressive PSO). Using the theory of stochastic differential and difference equations, we fully characterize the stability behavior of these algorithms. For well posed problems, a sufficient condition to achieve convergence is to select the PSO parameters close to the upper limit of second order stability. This result is also confirmed by numerical experimentation for different benchmark functions having an increasing degree of numerical difficulties. We also address how the discrete GPSO version (stability regions and trajectories) approaches the continuous PSO model as the time step decreases to zero. Finally, in the context of inverse problems, we address the question of how to select the appropriate PSO version: CP-PSO is the most explorative version and should be selected when we want to perform sampling of the posterior distribution of the inverse model parameters. Conversely, CC-PSO and GPSO provide higher convergence rates. Based on the analysis shown in this chapter, we can affirm that the PSO optimizers are not heuristic algorithms since there exist mathematical results that can be used to explain their consistency/convergence.