Online or face-to-face? An experimental study of examiner training

Abstract

Thousands of examiners are employed to mark candidate scripts from the suite of public examinations offered to students during the compulsory and post-compulsory schooling phases in England, Northern Ireland and Wales. All examiners undergo training to ensure that they interpret correctly, and apply consistently, the mark scheme for their particular paper. Traditionally this training has been delivered face-to-face, but several benefits of an online training environment have been identified. This paper outlines the findings of an experimental study designed to quantify and compare the effects of face-to-face and online training on experienced examiners' marking reliability. A sample of 89 experienced examiners was randomly allocated to either a face-to-face or online training group. The participants marked 30 General Certificate of Secondary Education History scripts prior to receiving training and another 30 scripts following training. Two measures of reliability were calculated: marking accuracy (absolute mark differences) and consistency (rank order correlations). The findings suggest that both modes of training had comparable, statistically significant positive effects on examiners' accuracy and consistency. The evidence indicates that online training may be an effective alternative to face-to-face training for the purposes of standardising examiners' marking. British Journal of Educational Technology © 2010 Becta.