Misleading Meta-Analyses during COVID-19 Pandemic: Examples of Methodological Biases in Evidence Synthesis

Abstract

Not all evidence is equal. Evidence-based public health and medicine emanate from the principle that there is a hierarchy of evidence, with systematic reviews and meta-analyses (SRMAs) being at the top, as the highest level of evidence. Despite this, it is common in literature to find SRMAs with methodological issues that can distort the results and can thus have serious public health or clinical implications. During the Coronavirus Disease 2019 (COVID-19) pandemic, the importance of evidence and the way in which evidence was produced was stress tested and revealed a wide array of methodological biases that might have led to misleading conclusions and recommendations. We provide a critical examination of methodological biases in selected SRMAs on COVID-19, which have been widely used to guide or justify some pharmaceutical and nonpharmaceutical interventions with high public health and clinical significance, such as mask wearing, asymptomatic transmission, and ivermectin. Through these selected examples, we highlight the need to address biases related to the methodological quality and relevance of study designs and effect size computations and considerations for critical appraisal of available data in the evidence synthesis process for better quality evidence. Such considerations help researchers and decision makers avoid misleading conclusions, while encouraging the provision of the best policy recommendations for individual and public health.