The effect of three commercial coaching courses on Step One USMLE performance

Abstract

Background: A substantial industry exists to provide formal review courses for Step 1 of the United States Medical Licensing Examination (USMLE). There are limited data on the usefulness of these courses. Aim: To determine whether or not student participation in a commercial coaching course improves performance on Step 1 of the USMLE. Methods: Scores achieved by 468 students on the National Board of Medical Examiners (NBME) Comprehensive Basic Science Examination (CBSE) were used to predict a score on Step 1 of the USMLE. The NBME is the organisation that prepares and administers the USMLE. Predicted USMLE scores were then regressed against the actual scores achieved by the students on Step 1. The students were divided into 2 groups: those who took a 3-4-week live commercial coaching course and those who studied on their own. Results: The regression lines for the 369 students who studied on their own and the 99 students who took a commercial coaching course were statistically indistinguishable. The analysis was powerful enough to have picked up a difference of 1% on average (P = 0.05) or 2 questions out of the 350 constituting Step 1 of the USMLE. Neither the students who performed above average nor those who performed below average on the GBSE improved their performance on Step 1 as a result of the coaching courses. Conclusions: Students who take a live, 3-4-week commercial coaching course to improve performance on Step 1 of the USMLE do not achieve higher scores than students who study on their own. Students should strongly consider whether or not a substantial investment in time and money for a commercial coaching course is justified in the light of such meagre returns.