Impact of a longitudinal faculty development program on the quality of multiple-choice question item writing in medical education

Abstract

Background: Like many other academic programs, medical education is incomplete without a robust assessment plan. Objective: The study aimed to evaluate the impact of longitudinal faculty development program (FDP) on the examination item quality (EIQ) from a cohort of medical college faculty members. Methods: Item analysis (IA) of multiple-choice questions (MCQs) from a cohort of medical tutors over a 3-year period (2017 [S1], 2018 [S2], and 2019 [S3]) before and following once-per-week FDP was conducted. The questions were from three randomly selected courses: man and his environment (MEV) from phase 1, central nervous system (CNS) from phase 2, and internal medicine (MED) from phase 3. Data assessed were 480 MCQs from the final exams in the courses. The parameters considered in IA were the difficulty index, index of discrimination, nonfunctional distractors (NFDs), distractor efficiency for each question item, and Cronbach's alpha (CA) for the test as a whole. Comparison over the 3 years was made using Fisher's exact test and repeated-measures ANOVA with Bonferroni test as post hoc test. Results: Overall, out of 480 MCQs, 272 had no NFD (52 [19.52%], 104 [38.24%], and 116 [42.65%] in 2017, 2018, and 2019, respectively) with a significant difference between S3, S2, and S1 (P < 0.0001). The mean CA for the exams in S1, S2, and S3, respectively, were 0.51, 0.77, and 0.84, P < 0.0001. Conclusion: There was an improvement in EIQ following the implementation of longitudinal FDP. Thus, the need for active training and retraining of the faculty for a better EIQ cannot be overemphasized.