Standard methods for constrained optimization

Abstract

Consider the general constrained optimization problem: (Formual Presented) The most simple and straightforward approach to handling constrained problems of the above form is to apply a suitable unconstrained optimization algorithm to a penalty function formulation of constrained problem. Unfortunately the penalty function method becomes unstable and inefficient for very large penalty parameter values if high accuracy is required. A remedy to this situation is to apply the penalty function method to a sequence of sub-problems, starting with moderate penalty parameter values, and successively increasing their values for the sub-problems. Alternatively, the Lagrangian function with associated necessary Karush-Kuhn-Tucker (KKT) conditions and duality serve to solve constrained problems that has led to the development of the Sequential Quadratic Programming (SQP) method that applies (Formual Presented) method to solve the KKT conditions.