Relationship Between Institutional Standardized Letter of Evaluation Global Assessment Ranking Practices, Interviewing Practices, and Medical Student Outcomes

Abstract

Background: Emergency medicine (EM) program directors rely largely on the standardized letter of evaluation (SLOE) to help determine which applicants to interview in the face of an increasing number of applications. To further characterize the SLOE's role in the EM application process, particularly the global assessment (GA) ranking and its effect on interviewing practices and medical student outcomes, the leaders of EM programs were surveyed regarding their experiences in both generating and utilizing the SLOE. Methods: Individuals on the Council of Emergency Medicine Residency Directors (CORD) and Clerkship Directors in Emergency Medicine (CDEM) Academy listservs were anonymously surveyed from March 21–30, 2015, with 18 questions in multiple-choice and fill-in-the-blank formats. Results: There were 99 respondents. Only 39 respondents (39%) reported adhering strictly to SLOE guidelines by evenly placing their students into thirds (top, middle, lower) on the SLOE GA. Most respondents interviewed individuals ranked in the lower third. Programs adhering strictly to ranking guidelines were more likely to interview students in the lower third than those adhering loosely or not at all. There was no relationship between a program's self-reported adherence to the SLOE ranking guidelines and the number of unmatched students in EM during the 2014 and 2015 academic years. Conclusion: Many SLOE writers do not strictly adhere to CORD's SLOE writing guidelines when using the GA ranking, due to the fear of adversely impacting an applicant's ability to successfully match into EM. This calls into question the validity of the SLOE as it is currently used. However, this study suggests that adhering to recommended SLOE ranking guidelines is unlikely to substantially increase the risk that students will fail to match. If more evaluators were to adhere to the guidelines, the SLOE could become the valid evaluation instrument that graduate medical education has long been pursuing.