A systematic review of reliability in corpus-based metadiscourse studies

Abstract

Metadiscourse has been a major focus of research over the last twenty-five years, attracting methodological approaches from the areas of textual pragmatics and discourse studies, many of which are supported by corpus linguistics. A major challenge in corpus-based discourse studies, however, is subjectivity, which may affect their quality and undermine their methodological rigor. To reduce subjectivity and guarantee consistency, assessing reliability of coding is essential. This study advocates combining quantitative with qualitative approaches to reliability. We argue that this mixed-method approach will provide a better assessment of reliability. To this aim, this methodological synthesis surveyed research covering empirical corpus-based studies on metadiscourse published in indexed and peer-reviewed journals. One major finding is that most studies did not report conducting any reliability measure. Issues in reliability accounts were also identified for those that did. Another major finding is a pervasive lack of transparency and comprehensiveness in reliability reports. Recommendations for enhancing reliability are listed.