A systematic appraisal of allegiance effect in randomized controlled trials of psychotherapy

Abstract

Background: Experimenter's allegiance (EA) refers to a personal confidence of the superiority of a specific psychotherapy treatment. This factor has been linked with larger treatment effects in favor of the preferred treatment. However, various studies have displayed contradictory results between EA and the pattern of treatment effects. Aims: Using a systematic approach followed by meta-analysis, we aimed to evaluate the impact of an allegiance effect on the results of psychotherapeutic studies. Method: We considered the meta-analyses of randomized controlled trials (RCTs) of different types of psychotherapies in the Cochrane Database of Systematic Reviews. Eligible articles included meta-analyses of RCTs with at least one study showing evidence of EA (i.e., allegiant study). Effect sizes in allegiant RCTs were compared with non-allegiant using random and fixed models and a summary relative odds ratio (ROR) were calculated. Heterogeneity was quantified with the I 2 metric. Results: A total of 30 meta-analyses including 240 RCTs were analyzed. The summary ROR was 1.31 [(95% confidence interval (CI: 1.03-1.66) P=0.30, I 2 =53%] indicating larger effects when allegiance exists. The impact of allegiance did not differ significantly (P>0.05) when we compared psychiatric versus medical outcomes. Allegiance effect was significant for all forms of psychotherapy except for cognitive behavioral therapy. Moreover, the impact of allegiance was significant only when the treatment integrity of delivered psychotherapy was not assessed. Allegiance effect was even stronger where the experimenter was also both the developer of the preferred treatment and supervised or trained the therapists. No significant differences were found between allegiant and non-allegiant studies in terms of overall quality of studies. Conclusions: Experimenter's allegiance influences the effect sizes of psychotherapy RCTs and can be considered non-financial conflict of interest introducing a form of optimism bias, especially since blinding is problematic in this kind of research. A clear reporting of EA in every single study should be given an opportunity to investigators of minimizing its overestimation effects.