The role of training in student examiner rating performance in a student-led mock OSCE

Abstract

Introduction: Peer assessments are increasingly prevalent in medical education, including student-led mock Objective Structured Clinical Examinations (OSCE). While there is some evidence to suggest that examiner training may improve OSCE assessments, few students undergo training before becoming examiners. We sought to evaluate an examiner training programme in the setting of a student-led mock OSCE. Methods: A year‑2 mock OSCE comprised of history taking (Hx) and physical examination (PE) stations was conducted involving 35 year‑3 (Y3) student examiners and 21 year‑5 (Y5) student examiners who acted as reference examiners. Twelve Y3 student-examiners attended an OSCE examiner training programme conducted by senior faculty. During the OSCE, Y3 and Y5 student examiners were randomly paired to grade the same candidates and scores were compared. Scores for checklist rating (CR) and global rating (GR) domains were assigned for both Hx and PE stations. Results: There was moderate to excellent correlation between Y3 and Y5 student examiners for both Hx (ICC 0.71–0.96) and PE stations (ICC 0.71–0.88) across all domains. For both Hx and PE stations, GR domain had poorer correlation than CR domains. Examiner training resulted in better correlations for PE but not Hx stations. Effect sizes were lower than the minimum detectible effect (MDE) sizes for all comparisons made. Discussion: Y3 student examiners are effective substitutes for Y5 student examiners in a Y2 mock OSCE. Our findings suggest that examiner training may further improve marking behaviour especially for PE stations. Further studies with larger sample sizes are required to further evaluate the effects of dedicated examiner training.