A Transformer-Based Deep Learning Model for Successful Predictions of the 2021 Second-Year La Niña Condition

Abstract

A purely data-driven and transformer-based model with a novel self-attention mechanism (3D-Geoformer) is used to make predictions by adopting a rolling predictive manner similar to that in dynamical coupled models. The 3D-Geoformer yields a successful prediction of the 2021 second-year cooling conditions that followed the 2020 La Niña event, including covarying anomalies of surface wind stress and three-dimensional (3D) upper-ocean temperature, the reoccurrence of negative subsurface temperature anomalies in the eastern equatorial Pacific and a corresponding turning point of sea surface temperature (SST) evolution in mid-2021. The reasons for the successful prediction with interpretability are explored comprehensively by performing sensitivity experiments with modulating effects on SST due to wind and subsurface thermal forcings being separately considered in the input predictors for prediction. A comparison is also conducted with physics-based modeling, illustrating the suitability and effectiveness of 3D-Geoformer as a new platform for El Niño and Southern Oscillation studies.