Does psychotherapy work? An umbrella review of meta-analyses of randomized controlled trials

Abstract

Objective: To map and evaluate the evidence across meta-analyses of randomized controlled trials (RCTs) of psychotherapies for various outcomes. Methods: We identified 173 eligible studies, including 247 meta-analyses that synthesized data from 5157 RCTs via a systematic search from inception to December 2016 in the PubMed, PsycINFO and Cochrane Database of Systematic Reviews. We calculated summary effects using random-effects models, and we assessed between-study heterogeneity. We estimated whether large studies had significantly more conservative results compared to smaller studies (small-study effects) and whether the observed positive studies were more than expected by chance. Finally, we assessed the credibility of the evidence using several criteria. Results: One hundred and ninety-nine meta-analyses were significant at P-value ≤ 0.05, and almost all (n = 196) favoured psychotherapy. Large and very large heterogeneity was observed in 130 meta-analyses. Evidence for small-study effects was found in 72 meta-analyses, while 95 had evidence of excess of significant findings. Only 16 (7%) provided convincing evidence that psychotherapy is effective. These pertained to cognitive behavioural therapy (n = 6), meditation therapy (n = 1), cognitive remediation (n = 1), counselling (n = 1) and mixed types of psychotherapies (n = 7). Conclusions: Although almost 80% meta-analyses reported a nominally statistically significant finding favouring psychotherapy, only a few meta-analyses provided convincing evidence without biases.