Judging the 'weight of evidence' in systematic reviews: Introducing rigour into the qualitative overview stage by assessing signal and noise

Abstract

The 'weight of evidence' in a topic area can be judged by assessing the 'Signal' from available research publications and tempering the importance attached by the level of 'Noise' (the inverse of methodological quality). This assessment process has validity and reliability and can be applied to the 'qualitative overview' stage of systematic reviews. This enables the important themes and areas of relevance to the research question to be identified. Important findings from individual papers may also be identified providing further information which may not be evident from quantitative analysis. The findings from these more qualitative stages of analysis complement, but do not replace, quantitative analysis.