Likelihood- and residual-based evaluation of medium-term earthquake forecast models for California

Abstract

Seven competing models for forecasting medium-termearthquake rates in California are quantitatively evaluated using the framework of the Collaboratory for the Study of Earthquake Predictability (CSEP). The model class consists of contrasting versions of the Every Earthquake a Precursor According to Size (EEPAS) and Proximity to Past Earthquakes (PPE) modelling approaches. Models are ranked by their performance on likelihood-based tests, which measure the consistency between a model forecast and observed earthquakes. To directly compare one model against another, we run a classical paired t-test and its non-parametric alternative on an information gain score based on the forecasts. These test scores are complemented by several residual-based methods,which offer detailed spatial information. The experiment period covers 2009 June-2012 September, when California experienced 23 earthquakes above the magnitude threshold. Though all models fail to capture seismicity during an earthquake sequence, spatio-temporal differences between models also emerge. The overall best-performing model has strong time- and magnitude-dependence, weights all earthquakes equally as medium-term precursors of larger events and has a full set of fitted parameters. Models with this time- and magnitude-dependence offer a statistically significant advantage over simpler baseline models. In addition, models that down-weight aftershocks when forecasting larger events have a desirable feature in that they do not overpredict following an observed earthquake sequence. This tendency towards overprediction differs between the simpler model, which is based on fewer parameters, and more complex models that include more parameters. © The Authors 2014. Published by Oxford University Press on behalf of The Royal Astronomical Society.