A new approach to testing forecast predictive accuracy

Abstract

The Diebold-Mariano test for predictive accuracy has been used widely and adapted for economic forecasts, but has not seen much activity in weather forecast verification. The technique is applied to both simulated verification sets as well as weather data at eight stations in Utah, and a loss function based on dynamic time warping (DTW) is used. Results of the simulation experiment show that the DTW technique can be useful if timing errors are the concern. Real test cases demonstrate the difficulty in automating some of the more advanced methods proposed here, but also show the utility in even the most basic test, which is an improvement over similar tests that do not account for temporal and/or contemporaneous correlation.