Assessing the quality of feedback in the peer-review process

Abstract

The feedback provided to authors by reviewers as part of a double-blind peer-review process was examined for two Australian conferences, one special international edition book and six international special edition journals (originating in the UK). The research sought to identify consistency of decision-making and the effectiveness of feedback for authors, in terms of the amount written and the tone of comments. The recommendation of acceptance or rejection of papers under the peer-review process is generally consistent, with reviewers agreeing with each other more often than they disagree. The feedback provided is mostly constructive and designed to help authors with rewrites and resubmissions. However, the amount of written commentary provided by reviewers is limited and in one-third of cases, the reviewers disagreed with each other, which generates additional work for the trackchairs and editors. The findings suggest that while imperfect, the process requires policy and managerial changes if good-quality reviews are to be encouraged.