Population and Gradient Based Optimization Techniques, a Theoretical Overview

Abstract

Many practical problems can be modeled only as a nonlinear continuous global optimization problem. It is usually impossible and impractical to solve them exactly. Evolutionary and hybrid algorithms are modern techniques to find optima for complex search spaces. This paper is a short summary about the optimization techniques, especially about the evolutionary based global searching algorithms, and the gradient based local searching algorithms. After the introduction, the second chapter gives a brief summary about population based techniques, especially about genetic and bacterial evolutionary algorithm, and particle swarm optimization. The next chapter discusses two gradient based searching technique, the steepest descent and the Levenberg-Marquardt method. This paper is a theoretical overview of the basics of my future Ph.D dissertation.