Assessing review reports of scientific articles: A literature review

Abstract

Computational support has been applied in different stages for automation of the peer review process, such as reviewer assignment to the article, review of content of the scientific article, detection of plagiarism and bias, all applying Machine Learning (ML) techniques. However, there is a lack of studies which identify the instruments used to evaluate the reviewers’ reports. This systematic literature review aims to find evidence about which techniques have been applied in the assessment of the reviewers’ reports. Therefore, six online databases were evaluated, in which 55 articles were identified, all published since 2000, meeting the inclusion criteria of this review. The result shows 6 relevant studies, which address models of assessment of scientific article reviews. Nevertheless, the use of ML was not identified in any case. Therefore, our findings demonstrate that there are a few instruments used to assess the reviewers’ reports and furthermore, they cannot be reliably used to extensively automate the review process.