Minimization and Least Squares

Abstract

Because Nature seems to strive for efficiency, many systems arising in physical applications are founded on a minimization principle. In a mechanical system, the stable equilibrium configurations minimize the potential energy. In an electrical circuit, the current adjusts itself to minimize the power. In optics and relativity, light rays follow the paths of minimal distance — the geodesics on the curved space-time manifold. Solutions to most of the boundary value problems arising in applications to continuum mechanics are also characterized by a minimization principle, which is then employed to design finite element numerical approximations to their solutions, [61, 81]. Optimization — finding minima or maxima — is ubiquitous throughout mathematical modeling, physics, engineering, economics, and data science, including the calculus of variations, differential geometry, control theory, design and manufacturing, linear programming, machine learning, and beyond.