Subtle skills: Using objective structured clinical examinations to assess gastroenterology fellow performance in system based practice milestones

Abstract

BACKGROUND System based practice (SBP) milestones require trainees to effectively navigate the larger health care system for optimal patient care. In gastroenterology training programs, the assessment of SBP is difficult due to high volume, high acuity inpatient care, as well as inconsistent direct supervision. Nevertheless, structured assessment is required for training programs. We hypothesized that objective structured clinical examination (OSCE) would be an effective tool for assessment of SBP. AIM To develop a novel method for SBP milestone assessment of gastroenterology fellows using the OSCE. METHODS For this observational study, we created 4 OSCE stations: Counseling an impaired colleague, handoff after overnight call, a feeding tube placement discussion, and giving feedback to a medical student on a progress note. Twenty-six first year fellows from 7 programs participated. All fellows encountered identical case presentations. Checklists were completed by trained standardized patients who interacted with each fellow participant. A report with individual and composite scores was generated and forwarded to program directors to utilize in formative assessment. Fellows also received immediate feedback from a faculty observer and completed a post-session program evaluation survey. RESULTS Survey response rate was 100%. The average composite score across SBP milestones for all cases were 6.22 (SBP1), 4.34 (SBP2), 3.35 (SBP3), and 6.42 (SBP4) out of 9. The lowest composite score was in SBP 3, which asks fellows to advocate for cost effective care. This highest score was in patient care 2, which asks fellows to develop comprehensive management plans. Discrepancies were identified between the fellows' perceived performance in their self-assessments and Standardized Patient checklist evaluations for each case. Eighty-seven percent of fellows agreed that OSCEs are an important component of their clinical training, and 83% stated that the cases were similar to actual clinical encounters. All participating fellows stated that the immediate feedback was “very useful.” One hundred percent of the fellows stated they would incorporate OSCE learning into their clinical practice. CONCLUSION OSCEs may be used for standardized evaluation of SBP milestones. Trainees scored lower on SBP milestones than other more concrete milestones. Training programs should consider OSCEs for assessment of SBP.