Evaluation of MAP D-PHASE heavy precipitation alerts in Switzerland during summer 2007

Abstract

This paper evaluates the performance of 13 mesoscale atmospheric models with respect to heavy precipitation alerts issued by these models in Switzerland during the summer 2007. All considered models contributed to an experimental real-time warning system, which was operated within the WWRP Forecast Demonstration Project MAP D-PHASE at that time. Seven of the considered models are deep convection resolving systems with grid spacings equal or less than 3 km. The models produced alerts if the predicted precipitation accumulation in alert regions (order of a few thousand square kilometers) exceeded the amount for an event with a return frequency of six per year. With an average probability of detection of 30% and a false alarm ratio of 60%, the accuracy of these alerts is poor but still skillful. An analysis of relative value reveals a clear tendency of deep convection resolving models to produce more useful alerts. The robustness of this finding with respect to the settings of the evaluation, the observational data set and the limited length of the evaluation period is successfully proven by sensitivity studies and bootstrap resampling experiments. The benefit of high resolution systems can partly be attributed to the higher update frequency of these models, even without any direct assimilation of precipitation observations. Probabilistic forecasts allow for a significant gain in relative value by choosing user dependent probability thresholds. A multi-model ensemble consisting of all deterministic models outperforms every deterministic model. Remarkably, a simple static recalibration of the best deterministic model performs equally well. © 2010 Elsevier B.V.