Systems for rating bodies of evidence used in systematic reviews of air pollution exposure and reproductive and children’s health: a methodological survey

Abstract

Background: Translating findings from systematic reviews assessing associations between environmental exposures and reproductive and children’s health into policy recommendations requires valid and transparent evidence grading. Methods: We aimed to evaluate systems for grading bodies of evidence used in systematic reviews of environmental exposures and reproductive/ children’s health outcomes, by conducting a methodological survey of air pollution research, comprising a comprehensive search for and assessment of all relevant systematic reviews. To evaluate the frameworks used for rating the internal validity of primary studies and for grading bodies of evidence (multiple studies), we considered whether and how specific criteria or domains were operationalized to address reproductive/children’s environmental health, e.g., whether the timing of exposure assessment was evaluated with regard to vulnerable developmental stages. Results: Eighteen out of 177 (9.8%) systematic reviews used formal systems for rating the body of evidence; 15 distinct internal validity assessment tools for primary studies, and nine different grading systems for bodies of evidence were used, with multiple modifications applied to the cited approaches. The Newcastle Ottawa Scale (NOS) and the Grading of Recommendations, Assessment, Development, and Evaluations (GRADE) framework, neither developed specifically for this field, were the most commonly used approaches for rating individual studies and bodies of evidence, respectively. Overall, the identified approaches were highly heterogeneous in both their comprehensiveness and their applicability to reproductive/children’s environmental health research. Conclusion: Establishing the wider use of more appropriate evidence grading methods is instrumental both for strengthening systematic review methodologies, and for the effective development and implementation of environmental public health policies, particularly for protecting pregnant persons and children.