DEFENDER: Detecting and forecasting epidemics using novel data-analytics for enhanced response

Abstract

In recent years social and news media have increasingly been used to explain patterns in disease activity and progression. Social media data, principally from the Twitter network, has been shown to correlate well with official disease case counts. This fact has been exploited to provide advance warning of outbreak detection, forecasting of disease levels and the ability to predict the likelihood of individuals developing symptoms. In this paper we introduce DEFENDER, a software system that integrates data from social and news media and incorporates algorithms for outbreak detection, situational awareness and forecasting. As part of this system we have developed a technique for creating a location network for any country or region based purely on Twitter data. We also present a disease nowcasting (forecasting the current but still unknown level) approach which leverages counts from multiple symptoms, which was found to improve the nowcasting accuracy by 37 percent over a model that used only previous case data. Finally we attempt to forecast future levels of symptom activity based on observed user movement on Twitter, finding a moderate gain of 5 percent over a time series forecasting model.