4D-Var Using Hessian Approximation and Backpropagation Applied to Automatically Differentiable Numerical and Machine Learning Models

Abstract

Constraining a numerical weather prediction (NWP) model with observations via 4D variational (4D-Var) Data assimilation (DA) is often difficult to implement due to the need to develop and maintain a software-based tangent linear model and adjoint model. One of the most common 4D-Var algorithms uses an incremental update procedure, which has been shown to be an approximation of the Gauss-Newton method. Here we demonstrate that when using a forecast model that supports flexible automatic differentiation, an efficient and in some cases more accurate alternative approximation of the Gauss-Newton method can be applied by combining backpropagation of errors with a Hessian approximation. This approach can be used with either a conventional physical model implemented with automatic differentiation or a machine learning (ML) based surrogate model. We test the new approach on a variety of Lorenz-96 and quasi-geostrophic models. The results indicate potential for a deeper integration of modeling, DA, and new technologies in a next-generation of operational forecast systems that leverage weather models designed to support flexible, on-the-fly automatic differentiation.