A large-scale assessment of temporal trends in meta-analyses using systematic review reports from the Cochrane Library

Abstract

Introduction: Previous studies suggest that many systematic reviews contain meta-analyses that display temporal trends, such as the first study's result being more extreme than later studies' or a drift in the pooled estimate. We assessed the extent and characteristics of temporal trends using all Cochrane intervention reports published 2008-2012. Methods: We selected the largest meta-analysis within each report and analysed trends using methods including a Z-test (first versus subsequent estimates); generalised least squares; and cumulative sum charts. Predictors considered include meta-analysis size and review group. Results: Of 1288 meta-analyses containing at least 4 studies, the point estimate from the first study was more extreme and in the same direction as the pooled estimate in 738 (57%), with a statistically significant difference (first versus subsequent) in 165 (13%). Generalised least squares indicated trends in 717 (56%); 18% of fixed effects analyses had at least one violation of cumulative sum limits. For some methods, meta-analysis size was associated with temporal patterns and use of a random effects model, but there was no consistent association with review group. Conclusions: All results suggest that more meta-analyses demonstrate temporal patterns than would be expected by chance. Hence, assuming the standard meta-analysis model without temporal trend is sometimes inappropriate. Factors associated with trends are likely to be context specific.