Social sensing of high-impact rainfall events worldwide: A benchmark comparison against manually curated impact observations

Abstract

Impact-based weather forecasting and warnings create the need for reliable sources of impact data to generate and evaluate models and forecasts. Here we compare outputs from social sensing - analysis of unsolicited social media data, in this case from Twitter - against a manually curated impact database created by the Met Office. The study focuses on high-impact rainfall events across the globe between January-June 2017. Social sensing successfully identifies most high-impact rainfall events present in the manually curated database, with an overall accuracy of 95 %. Performance varies by location, with some areas of the world achieving 100 % accuracy. Performance is best for severe events and events in English-speaking countries, but good performance is also seen for less severe events and in countries speaking other languages. Social sensing detects a number of additional high-impact rainfall events that are not recorded in the Met Office database, suggesting that social sensing can usefully extend current impact data collection methods and offer more complete coverage. This work provides a novel methodology for the curation of impact data that can be used to support the evaluation of impact-based weather forecasts.