Magnitude of effects in clinical trials published in high-impact general medical journals

Abstract

Background: Prestigious journals select for publication studies that are considered most important and informative. We aimed to examine whether high-impact general (HIG) medical journals systematically demonstrate more favourable results for experimental interventions compared with the rest of the literature. Methods: We scrutinized systematic reviews of the Cochrane Database (Issue 4, 2009) and meta-analyses published in four general journals (2008-09). Eligible articles included ≥1 binary outcome meta-analysis(es) pertaining to effectiveness with ≥1 clinical trial(s) published in NEJM, JAMA or Lancet. Effect sizes in trials from NEJM, JAMA or Lancet were compared with those from other trials in the same meta-analyses by deriving summary relative odds ratios (sRORs). Additional analyses examined separately early- and late-published trials in HIG journals and journal-specific effects. Results: A total of 79 meta-analyses including 1043 clinical trials were analysed. Trials in HIG journals had similar effects to trials in other journals, when there was large-scale evidence, but showed more favourable results for experimental interventions when they were small. When HIG trials had less than 40 events, the sROR was 1.64 [95% confidence interval (95% CI): 1.23-2.18). The difference was most prominent when small early trials published in HIG journals were compared with subsequent trials [sROR 2.68 (95% CI: 1.33-5.38)]. Late-published HIG trials showed no consistent inflation of effects. The patterns did not differ beyond chance between NEJM, JAMA or Lancet. Conclusions: Small trials published in the most prestigious journals show more favourable effects for experimental interventions, and this is most prominent for early-published trials in such journals. No effect inflation is seen for large trials. Published by Oxford University Press on behalf of the International Epidemiological Association © The Author 2011; all rights reserved.