On the Relationship Between Reliability Diagrams and the “Signal-To-Noise Paradox”

Abstract

The “signal-to-noise paradox” for seasonal forecasts of the winter North Atlantic Oscillation (NAO) is often described as an “underconfident” forecast and measured using the ratio-of-predictable components (RPCs) metric. However, comparison of RPC with other measures of forecast confidence, such as spread-error ratios, can give conflicting impressions, challenging this informal description. We show, using a linear statistical model, that the “paradox” is equivalent to a situation where the reliability diagram of any percentile forecast has a slope exceeding 1. The relationship with spread-error ratios is shown to be far less direct. We furthermore compute reliability diagrams of winter NAO forecasts using seasonal hindcasts from the European Centre for Medium-range Weather Forecasts and the UK Meteorological Office. While these broadly exhibit slopes exceeding 1, there is evidence of asymmetry between upper and lower terciles, indicating a potential violation of linearity/Gaussianity. The limitations and benefits of reliability diagrams as a diagnostic tool are discussed.