New machine-marked tests for selection into core medical training: Evidence from two validation studies

Abstract

This study examined whether two machine-marked tests (MMTs; a clinical problem-solving test and situational judgement test), previously validated for selection into UK general practice (GP) training, could provide a valid methodology for shortlisting into core medical training (CMT). A longitudinal design was used to examine the MMTs' psychometric properties in CMT samples, and correlations between MMT scores and CMT interview outcomes. Independent samples from two years were used: in 2008, a retrospective analysis was conducted (n=1,711), while in 2009, CMT applicants completed the MMTs for evaluation purposes (n=2,265). Both MMTs showed good reliability in CMT samples, similar to GP samples. Both MMTs were good predictors of CMT interview performance (r=0.56, p<0.001 in 2008; r=0.61, p<0.001 in 2009) and offered incremental validity over the current shortlisting process. The GP MMTs offer an appropriate measurement methodology for selection into CMT, representing a significant innovation for selection methodology. © Royal College of Physicians, 2009. All rights reserved.