Variable metric or quasi-Newton methods are well known and commonly used in connection with unconstrained optimization, since they have good theoretical and practical convergence properties. Although these methods were originally developed for small- and moderate-size dense problems, their modifications based either on sparse, partitioned or limited-memory updates are very efficient on large-scale sparse problems. Very significant applications of these methods also appear in nonlinear least-squares approximation and nonsmooth optimization. In this contribution, we give an extensive review of variable metric methods and their use in various optimization fields.
Lukšan, L., & Spedicato, E. (2000). Variable metric methods for unconstrained optimization and nonlinear least squares. Journal of Computational and Applied Mathematics, 124(1–2), 61–95. https://doi.org/10.1016/S0377-0427(00)00420-9