A Comparison of Several Current Optimization Methods, and the use of Transformations in Constrained Problems

Abstract

The performances of eight current methods for unconstrained optimization are evaluated using a set of test problems with up to twenty variables. The use of optimization techniques in the solution of simultaneous non-linear equations is also discussed. Finally transformations whereby inequality constraints of certain forms can be eliminated from the formulation of an optimization problem are described, and examples of their use compared with other methods for handling such constraints. 1. Introduction The type of optimization problem with which this paper is concerned is that in which the objective function is highly non-linear and the number of independent variables is small, with a maximum of about twenty. The conclusions therefore do not apply to either linear or mildly non-linear problems. For linear problems, several thousand independent variables can be handled, whilst for mildly non-linear problems the possible number of independent variables is several hundred. The first part of this paper describes experimental comparisons of the performances of eight current methods for unconstrained optimization based on a set of test functions with up to twenty independent variables. Little comparative information on these methods has been published, understandably since several of the methods are very recent; in particular, little has been reported on the use of these methods with as many as twenty variables. There follows a brief demonstration that the problem of solving sets of simultaneous non-linear equations should be attempted by using methods based on the Jacobian, rather than by merely minimizing the sum of squared residuals. The paper then goes on to consider transformations by which problems of constrained optimization (only inequality constraints of certain forms will be considered) can be reduced to a form in which no constraints explicitly appear, so that they are then suitable for solution by methods incapable of handling constraints. These latter methods include several which are more powerful than the limited number available for con-strained optimization of a general non-linear function. Some numerical experiments in the use of this approach in conjunction with one of the more efficient algorithms for unconstrained optimization are described, and the results compared with certain other techniques for constrained optimization.