Regularized Dawid–Sebastiani score for multivariate ensemble forecasts

Abstract

The Dawid–Sebastiani score (DSS) is a theoretically attractive tool for evaluating the accuracy of multivariate ensemble forecasts, wherein each ensemble member produces a vector of forecasts. It has been little used to date mostly because its computation requires calculation and inversion of the ensemble covariance matrix for each forecast occasion. These matrices are not invertible unless the ensemble size is larger than the dimension of the forecast vectors. Moreover, these matrices are poorly estimated unless the ensemble size is quite large relative to the dimension of the vectors. This article describes application to the inverse correlation matrix in the DSS of a relatively recently developed regularization procedure called the graphical lasso (“glasso”), which not only suppresses the sampling variability of the score for relatively small ensemble sizes but also allows its computation even when the ensemble size is smaller than the dimension of the forecast and observation vectors. Use of glasso regularization with the DSS, and its performance in comparison to the Energy Score and the Variogram Score, are illustrated using a novel statistical model for multivariate ensemble forecasts which allows separate manipulation of bias, univariate calibration, multivariate calibration, and multivariate correlation characteristics.