Issues in comparisons between meta-analyses and large trials

  • Ioannidis J
  • Cappelleri J
  • Lau J
  • 66

    Readers

    Mendeley users who have this article in their library.
  • 197

    Citations

    Citations of this article.

Abstract

CONTEXT The extent of concordance between meta-analyses and large trials on the same topic has been investigated with different protocols. Inconsistent conclusions created confusion regarding the validity of these major tools of clinical evidence. OBJECTIVE To evaluate protocols comparing meta-analyses and large trials in order to understand if and why they disagree on the concordance of these 2 clinical research methods. DESIGN Systematic comparison of protocol designs, study selection, definitions of agreement, analysis methods, and reported discrepancies between large trials and meta-analyses. RESULTS More discrepancies were claimed when large trials were selected from influential journals (which may prefer trials disagreeing with prior evidence) than from already performed meta-analyses (which may target homogeneous trials) and when both primary and secondary (rather than only primary) end points were considered. Depending on how agreement was defined, kappa coefficients varied from 0.22 (low agreement) to 0.72 (excellent agreement). The correlation of treatment effects between large trials and meta-analyses varied from -0.12 to 0.76, but was more similar (0.50-0.76) when only primary end points were considered. When both the magnitude and uncertainty of treatment effects were considered, large trials disagreed with meta-analyses 10% to 23% of the time. Discrepancies were attributed to different disease risks, variable protocols, quality, and publication bias. CONCLUSIONS Comparisons of large trials with meta-analyses may reach different conclusions depending on how trials and meta-analyses are selected and how end points and agreement are defined. Scrutiny of these 2 major research methods can enhance our appreciation of both for guiding medical practice.

Get free article suggestions today

Mendeley saves you time finding and organizing research

Sign up here
Already have an account ?Sign in

Find this document

Authors

Cite this document

Choose a citation style from the tabs below

Save time finding and organizing research with Mendeley

Sign up for free