Extending coarsened exact matching to multiple cohorts: an application to longitudinal well-being program evaluation within an employer population

Abstract

Research to date within the field of well-being program evaluation has considered the study population to have either been given a treatment or not, and that matching will yield an unbiased, efficient estimate of the treatment causal effect. As well-being intervention programs become more sophisticated and diverse in their offerings, so too must the methods for assessing program effect. The objective of this research was to extend the traditional binary cohort assignment in quasi-experimental program evaluation in order to quantify the differential effects of a multi-tiered well-being improvement program administered over a 3 year period in a large employer. Data collected over this 3 year period included well-being assessments and medical claims from 17,669 employees and spouses. These individuals were assigned different cohorts based on intervention program intensity and matched utilizing coarsened exact matching. The matching process was able to remove 85 %, on average, of detectable bias across all comparison cohorts. A weighted generalized linear model, using the coarsened exact matching derived weights, was estimated to quantify the net (difference-in-difference) causal effect of the well-being intervention program. The results showed an increase of overall well-being on average in the High Intensity cohort of 1.48 and 1.32 points in the Mild Intensity cohort. The non-intervened cohort only evidenced a 0.57 point increase in overall well-being. The methodology reported here provides an expanded and robust approach to matching on different cohorts for the purpose of program evaluation.