Identification of Real and Artifactual Moderators of Effect Size in Meta-Analysis

Abstract

This article argues that while meta-analytic studies are widely used in psychological literature, heterogeneity and the potential for confounding remain major problems in the interpretation of meta-analytic study results. The article demonstrates the use of exploratory analysis including graphical methods prior to meta-analysis, and introduces a methodology to screen for artifactual effects. These procedures are illustrated on effect size data comparing depression treatment outcome from psychotherapy versus pharmacotherapy. Results support prior findings of a nonsignificant difference in effect size between the two treatments. They also support findings that treatment type accounts for only a very small proportion of outcome variance. However, the results indicate that some previously reported covariates of depression treatment outcome may be artifactual.