The WMO challenge to develop and demonstrate the best new user-oriented forecast verification metric

Abstract

The public, industry, emergency managers and other decision makers can use weather, climate and impact forecasts more effectively in their decision making when the quality of forecasts is measured in terms that are meaningful and comprehensible to them. To encourage the development of user-oriented verification approaches and support the major projects of the World Meteorological Organization (WMO) World Weather Research Program, a challenge was issued to develop and demonstrate new user-oriented forecast verification metrics. Several new forecast verification metrics were proposed to meet the needs of very different user-communities including public safety and amenity, shipping, aviation, defence, agriculture, and water resources. A few general purpose metrics were also proposed. The winner of the inaugural verification challenge proposed a new metric called the Spatial Probability Score for assessing forecasts for the location of a relevant boundary or contour, for example, sea ice edge or extent of flood inundation. We hope and expect that many of the user-oriented forecast verification metrics submitted to the inaugural verification challenge will be taken up by the broader community.