The use of a formative OSCE to prepare emergency medicine residents for summative OSCEs: a mixed-methods cohort study

Abstract

Background: The objective structured clinical examination (OSCE) is a part of emergency medicine (EM) examinations such as the Masters of Medicine in Emergency Medicine (MMed) examination and the equivalent Member of the Royal College of Emergency Medicine (MRCEM) examination. The use of formative OSCEs to prepare EM residents for summative OSCEs has not been investigated. This study aimed to evaluate the role of formative OSCEs in preparing EM residents for the MMed and MRCEM OSCE. Methods: This was an observational, retrospective, mixed-methods cohort study. We analysed data from formative OSCEs conducted by the National Healthcare Group EM residency programme from 2013 to 2019, and from a questionnaire distributed to all residents during the study period. Residents’ formative OSCE participation and scores were compared with first-attempt summative OSCE success. Qualitative analysis of residents’ opinions on the usefulness of the formative OSCE was performed. Results: Forty-three of the 50 (86.0%) residents attended at least one formative OSCE. Of the 46 who responded to the questionnaire, 40 (87.0%) had attempted and succeeded in the MMed or MRCEM OSCE, of whom 35 (87.5%) had succeeded on the first attempt. Residents who succeeded in the summative OSCE on the first attempt tended to have higher proximate (mean = 70.6, SD = 8.9 vs mean = 64.3, SD = 10.8) and mean (mean = 67.4, SD = 7.1 vs mean = 62.8, SD = 7.3) formative OSCE scores. All 8/40 (20.0%) residents who attended more than three formative OSCEs succeeded in the summative OSCE on their first attempt. Residents’ formative OSCE scores tended to improve with successive formative OSCEs, demonstrating a positive training effect. All residents felt that the formative OSCE was useful in preparing them for the summative OSCE. Conclusions: Participation in multiple formative OSCEs was beneficial in preparing residents for the summative OSCE. The formative OSCE was useful in familiarising residents with the examination, giving them an opportunity to perform in near-examination conditions, and providing feedback to residents and faculty about their progress. Our findings may support the implementation of formative OSCEs in other training programmes to prepare learners for high-stake summative OSCEs.