Improving the Quality of MCQs by Enhancing Cognitive Level and using Psychometric Analysis

Abstract

Faculty development programs are an essential part of any reputable institution. Faculty training through various conferences and workshops will help develop competencies for high quality assessment. Objective: To evaluate the effectiveness of ongoing faculty training initiatives aimed at improving the standard of MCQ development. Methods: Faculty members of FRPMC were engaged in this observational, quasi-experimental study. Convenient sampling was done. Three different questionnaires were provided to the participants. The participants were tasked with accurately identifying flaws, cognitive levels in the pre- and post-test questionnaire items, as well as post hoc analysis to discard or modify the Items. Results: Items with multiple flaws were assessed with mean value in pre-test score was 1.349 compared to post-test score 3.442 which were statistically significant (p-value <0.05). The pre- and post-test questionnaire to correctly identify the cognitive levels showed pre-test 77.5% compared to 87.5% participants identified correct cognitive level in post-test. In post hoc analysis, participants were able to identify 5 questions out of 7 which need to be discarded and 3 questions out of 5 which needed improvement according to the key given to them. Conclusions: Our study revealed that workshop helped faculty identify item flaws with significant improvement in pre- and post-test scores. The cognitive capabilities of faculty were improved in all three levels of cognition, with greatest improvements shown in higher complexity questions (C3). There was a noticeable improvement in knowledge and the participant's capacity to identify and correct errors.