Making causal inferences about treatment effect sizes from observational datasets

Abstract

In the era of big data and cloud computing, analysts need statistical models to go beyond predicting outcomes to forecasting how outcomes change when decision-makers intervene to change one or more causal factors. This paper reviews methods to estimate the causal effects of treatment choices on patient health outcomes using observational datasets. Methods are limited to those that model choice of treatment (propensity scoring) and treatment outcomes (instrumental variable, difference in differences, control function). A regression framework was developed to show how unobserved confounding covariates and heterogeneous outcomes can introduce biases to effect size estimates. In response to criticisms that outcome approaches are not systematic and subject to model misspecification error, we extend the control function approach of Lu and White by applying Best Approximating Model technology (BAM-CF). Results from simulation experiments are presented to compare biases between BAM-CF and propensity scoring in the presence of an unobserved confounder. We conclude no one strategy is ‘optimal’ for all datasets, and analyst should consider multiple approaches to assess robustness. For both observational and randomized datasets, researchers should assess how moderating covariates impact estimates of treatment effect sizes so that clinicians can understand what is best for each individual patient.