Inclusive critical appraisal of qualitative and quantitative findings in evidence synthesis

Abstract

A diversity of approaches for critically appraising qualitative and quantitative evidence exist and emphasize different aspects. These approaches lack clear processes to facilitate rating the overall quality of the evidence for aggregated findings that combine qualitative and quantitative evidence. We draw on a meta-aggregation of implementation and process evaluations to illustrate a method for critically appraising empirical findings generated from qualitative and quantitative studies. This method includes a rubric for standardizing assessments of the overall quality of evidence in an evidence synthesis or mixed-method systematic review. The method first assesses the credibility of each finding extracted from a study. These individual assessments then feed into an overall score for any synthesized finding generated from the meta-aggregation. We argue that this approach provides a balanced and inclusive method of critical appraisal by first assessing individual findings, rather than studies, using flexible criteria applicable to a range of primary study methods to derive an overall assessment of synthesized findings.