Anti-racism in CanMEDS 2025

Abstract

Background: The purpose of this study was to investigate the use of item analysis to assess objectively the quality of items on the Calgary-Cambridge Communications OSCE checklist. Methods: A total of 150 first year medical students were provided with extensive teaching on the use of the Calgary-Cambridge Guidelines for interviewing patients and participated in a final year end 20 minute communication OSCE station. Grouped into either the upper half (50%) or lower half (50%) communication skills performance groups, discrimination, difficulty and point biserial values were calculated for each checklist item. Results: The mean score on the 33 item communication checklist was 24.09 ( SD = 4.46) and the internal reliability coefficient was ? = 0.77. Although most of the items were found to have moderate ( k = 12, 36%) or excellent ( k = 10, 30%) discrimination values, there were 6 (18%) identified as ‘fair’ and 3 (9%) as ‘poor’. A post-examination review focused on item analysis findings resulted in an increase in checklist reliability ( ? = 0.80). Conclusions: Item analysis has been used with MCQ exams extensively. In this study, it was also found to be an objective and practical approach to use in evaluating the quality of a standardized OSCE checklist.