Choosing the Optimal Numerical Precision for Data Assimilation in the Presence of Model Error

Abstract

The use of reduced numerical precision within an atmospheric data assimilation system is investigated. An atmospheric model with a spectral dynamical core is used to generate synthetic observations, which are then assimilated back into the same model using an ensemble Kalman filter. The effect on the analysis error of reducing precision from 64 bits to only 22 bits is measured and found to depend strongly on the degree of model uncertainty within the system. When the model used to generate the observations is identical to the model used to assimilate observations, the reduced-precision results suffer substantially. However, when model error is introduced by changing the diffusion scheme in the assimilation model or by using a higher-resolution model to generate observations, the difference in analysis quality between the two levels of precision is almost eliminated. Lower-precision arithmetic has a lower computational cost, so lowering precision could free up computational resources in operational data assimilation and allow an increase in ensemble size or grid resolution.