Peer reviewer training to build capacity in engineering education research

Abstract

The Engineering Education Research (EER) Peer Review Training (PERT) project aimed to develop EER scholars’ peer review skills through mentored experiences reviewing journal manuscripts. Concurrently, the project explored how EER scholars develop capabilities for evaluating and conducting EER scholarship through peer reviewing. PERT used a mentoring structure in which two researchers with little reviewing experience were paired with an experienced mentor to complete three manuscript reviews collaboratively. Using a variety of techniques including think aloud protocols, structured peer reviews, and exit surveys, the PERT research team addressed the following research questions: (1) To what extent are the ways in which reviewers evaluate manuscripts influenced by reviewers’ varied levels of expertise? and (2) To what extent does participation in a mentored peer reviewer programme influence reviewers’ EER manuscript evaluations? Data were collected from three cohorts of the mentored review programme over 18 months. Findings indicate that experience influenced reviewers’ evaluation of EER manuscripts at the start of the programme, and that participation can improve reviewers’ understanding of EER disciplinary conventions and their connection to the EER community. Deeper understanding of the epistemological basis for manuscript reviews may reveal ways to strengthen professional preparation in engineering education as well as other disciplines.