The Effects of Heterogeneity in the Comparative Effectiveness of Individual Treatments in Randomised Trials

Abstract

In some randomised trials, the new treatment can be compared with usual care which can include multiple treatments and result in a heterogeneous control group. In this paper, we use simulation to assess the performance of various statistical methods to infer the individual effects of the various control treatments. These methods include inverse Probability Weighting, Doubly Robust Inverse Probability Weighting, Propensity Score, Disease Risk Score, Standardization and Multivariable Logistic Regression. Different scenarios were tested including unmeasured heterogeneity with or without confounding. The methods perform well when heterogeneity and confounding are both fully captured; however, for the scenarios where heterogeneity is not fully captured this leads to biased effect estimates, particularly where there is also unobserved confounding. Thus, leading to potentially misleading comparative effectiveness of individual treatments.