Assessing and reporting the quality of commercial weather forecasts

Abstract

In 2005 the Royal Meteorological Society commissioned a study to examine current issues regarding the quality (fitness for purpose) of commercial weather forecasts in the United Kingdom. UK commercial weather forecast providers and users were consulted by means of on-line questionnaires, interviews, visits and an open workshop discussion. Results from this consultation uncovered significant deficiencies in the methodologies and in the communication of forecast quality assessments, a lack of open dialogue and transparency in the industry, and that some users may be indifferent to forecast quality. Descriptive or worded forecasts cannot be assessed objectively. However, suitable quality assessment methods are available for nearly all types of quantitative forecasts identified in the consultation. The crucial importance of choosing proper quality assessment metrics, the impact of their statistical properties on results and the need to estimate the statistical significance of quality assessment results were exemplified by means of four case studies, one of which is presented in this paper. The findings from this study have led to a set of practical recommendations aiming to establish the discipline and rigour that are necessary for achieving best practice in the quality assessment of weather forecasts. Specific recommendations were also made to the Royal Meteorological Society to set up a special commission that would promote a sense of community within the industry, and to run an accreditation scheme that would encourage best practice on a voluntary basis. Copyright © 2008 Royal Meteorological Society.