Treatment effect bias in randomised controlled trials using surrogate outcomes: a preliminary cohort study analysis

Abstract

Clinical Trials Methodology Conference; 2011 Oct 4-5; Bristol Marriott City Centre, Bristol, UK. Background: Ideally, decisions on the value of health technologies should be based on evidence from well-conducted clinical trials that assess clinically important final patient-relevant outcomes, such as mortality or impaired quality of life. Pressure to reduce the delay in the availability of technologies to patients has led to an increased reliance on the use of surrogate outcomes [1]. A key tenant of surrogate outcomes is unbiased quantification of the predictive treatment effect on the final patient-relevant clinical outcomes. This study compares the treatment outcome of trials that use a surrogate versus a final patient-relevant primary outcome. Materials and methods: We searched for all randomized controlled trials (RCTs) published in JAMA, NEJM, Lancet, BMJ, PLoS Medicine and Annals of Internal Medicine in 2005 and 2006 [2]. We distinguished between trials that used a surrogate or a patient-relevant primary outcome. An outcome was defined as a surrogate if it did not directly measure "how a patient feels, functions, or survives" [3] or was judged to be a substitute and predictive of a final outcome [1]. We excluded non-RCTs, composite (of both surrogate and final) outcomes, economic evaluations and noninterventional technologies. Surrogate and final outcome trials were matched on patient population, intervention, journal and year of publication. In this preliminary analysis we compare the two groups of trials based on the statistical significance and direction of their outcome results. Results: Of the 639 citations identified by our searches, we included 138 trials that used a primary surrogate outcome ('surrogate trials') and 132 trials that used a final patient-relevant outcomes ('final trials'). Table 1 summarises the trial characteristics used for matching. Other trial characteristics also appeared to be well balanced except for the length of follow-up (i.e. more studies with follow up <30 days and >1 year for final trials). Conclusions: This preliminary analysis supports our initial hypothesis that the use of surrogate outcomes is more likely to lead statistically significant treatment effects than patient-relevant primary outcomes. We are currently undertaking additional analysis using actual effect sizes in meta-analytic/ meta-regression framework. These results have important implications for payers faced with making coverage decisions on the effectiveness and cost-effectiveness of new treatments based on surrogate rather than final clinical trials data