Using a Commercially Available Web-Based Evaluation System to Enhance Residents' Teaching

Abstract

BACKGROUND: Residents-as-teachers (RATs) programs have been shown to improve trainees' teaching skills, yet these decline over time.INTERVENTION: We adapted a commercial Web-based system to maintain resident teaching skills through reflection and deliberate practice and assessed the system's ability to (1) prevent deterioration of resident teaching skills and (2) provide information to improve residents' teaching skills and teaching program quality.METHODS: Ten first-year obstetrics-gynecology (Ob-Gyn) residents participated in a RATs program. Following the program, they used a commercial evaluation system to complete self-assessments of their teaching encounters with medical students. Students also evaluated the residents. To assess the system's effectiveness, we compared these residents to historical controls with an Objective Structured Teaching Examination (OSTE) and analyzed the ratings and the free text comments of residents and students to explore teaching challenges and improve the RATs program.RESULTS: The intervention group outscored the control group on the OSTE (mean score ± SD  =  81 ± 8 versus 74 ± 7; P  =  .05, using a 2-tailed Student t-test). Rating scale analysis showed resident self-assessments were consistently lower than student evaluations, with the difference reaching statistical significance in 3 of 6 skills (P < .05). Comments revealed that residents most valued using innovative teaching techniques, while students most valued a positive educational climate and interpersonal connections with residents. Recommended targets for RATs program improvement included teaching feedback, time-limited teaching, and modeling professionalism behaviors.CONCLUSIONS: Our novel electronic Web-based reinforcement system shows promise in preventing deterioration of resident teaching skills learned during an Ob-Gyn RATs program. The system also was effective in gaining resident and student insights to improve RATs programs. Because our intervention was built upon a commercially available program, our approach could prove useful to the large population of current subscribers.