Objectives Between-study heterogeneity plays an important role in random-effects models for meta-analysis. Most clinical trials are small, and small trials are often associated with larger effect sizes. We empirically evaluated whether there is also a relationship between trial size and heterogeneity (τ). Study Design and Setting We selected the first meta-analysis per intervention review of the Cochrane Database of Systematic Reviews Issues 2009-2013 with a dichotomous (n = 2,009) or continuous (n = 1,254) outcome. The association between estimated τ and trial size was evaluated across meta-analyses using regression and within meta-analyses using a Bayesian approach. Small trials were predefined as those having standard errors (SEs) over 0.2 standardized effects. Results Most meta-analyses were based on few (median 4) trials. Within the same meta-analysis, the small study τ<inf>S</inf><sup>2</sup> was larger than the large-study τ<inf>L</inf><sup>2</sup> [average ratio 2.11; 95% credible interval (1.05, 3.87) for dichotomous and 3.11 (2.00, 4.78) for continuous meta-analyses]. The imprecision of τ<inf>S</inf> was larger than of τ<inf>L</inf>: median SE 0.39 vs. 0.20 for dichotomous and 0.22 vs. 0.13 for continuous small-study and large-study meta-analyses. Conclusion Heterogeneity between small studies is larger than between larger studies. The large imprecision with which τ is estimated in a typical small-studies' meta-analysis is another reason for concern, and sensitivity analyses are recommended.
Inthout, J., Ioannidis, J. P. A., Borm, G. F., & Goeman, J. J. (2015). Small studies are more heterogeneous than large ones: A meta-meta-analysis. Journal of Clinical Epidemiology, 68(8), 860–869. https://doi.org/10.1016/j.jclinepi.2015.03.017