Comparing modelling approaches for the estimation of government intervention effects in COVID-19: Impact of voluntary behavior changes

Abstract

The efficacy of government interventions in epidemic has become a hot subject since the onset of COVID-19. There is however much variation in the results quantifying the effects of interventions, which is partly related to the varying modelling approaches employed by existing studies. Among the many factors affecting the modelling results, people’s voluntary behavior change is less examined yet likely to be widespread. This paper therefore aims to analyze how the choice of modelling approach, in particular how voluntary behavior change is accounted for, would affect the intervention effect estimation. We conduct the analysis by experimenting different modelling methods on a same data set composed of the 500 most infected U.S. counties. We compare the most frequently used methods from the two classes of modelling approaches, which are Bayesian hierarchical model from the class of computational approach and difference-in-difference from the class of natural experimental approach. We find that computational methods that do not account for voluntary behavior changes are likely to produce larger estimates of intervention effects as assumed. In contrast, natural experimental methods are more likely to extract the true effect of interventions by ruling out simultaneous behavior change. Among different difference-in-difference estimators, the two-way fixed effect estimator seems to be an efficient one. Our work can inform the methodological choice of future research on this topic, as well as more robust re-interpretation of existing works, to facilitate both future epidemic response plans and the science of public health.