The ensemble-adjusted Ignorance Score for forecasts issued as normal distributions

Abstract

This study considers the application of the Ignorance Score (IS, also known as the Logarithmic Score) for ensemble verification. In particular, we consider the case where an ensemble forecast is transformed to a normal forecast distribution, and this distribution is evaluated by the IS. It is shown that the IS systematically depends on the ensemble size, such that larger ensembles yield better expected scores. An ensemble-adjusted IS is proposed, which extrapolates the score of an m-member ensemble to the score that the ensemble would achieve if it had fewer or more than m members. Using the ensemble adjustment, a fair version of the IS is derived, which is optimized if ensembles are statistically consistent with the observations. The benefit of the ensemble adjustment is illustrated by comparing ISs of ensembles of different sizes in a seasonal climate forecasting context and a medium-range weather forecasting context. An ensemble-adjusted score can be used for a fair comparison between ensembles of different sizes, and to accurately estimate the expected score of a large operational ensemble by running a much smaller hindcast ensemble.