Propensity score analysis: Recent debate and discussion

Abstract

Propensity score analysis is often used to address selection bias in program evaluation with observational data. However, a recent study suggested that propensity score matching may accomplish the opposite of its intended goal—in-creasing imbalance, inefficiency, model dependence, and bias. We assess common propensity score models and offer our responses to these criticisms. We used Monte Carlo methods to simulate two alternative settings of data creation—selection on observed variables versus selection on unobserved variables—andcomparedeightpro-pensity score models on bias reduction and sample-size retention. Based on the simulations, no single propensity score method reduced bias across all scenarios. Optimal results depend on the fit between assumptions embedded in the analytic model and the process of data generation. Methodologic knowledge of model assumptions and substantive knowledge of causal mechanisms, including sources of selection bias, should inform the choice of analytic strategies involving propensity scores.