Performance of Chat Generative Pretrained Transformer and Bard on the Questions Asked in the Dental Specialty Entrance Examination in Turkey Regarding Bloom’s Revised Taxonomy

Abstract

Objective: This study aimed to compare the performance of chat generative pretrained transformer (ChatGPT) (GPT-3.5) and Bard, 2 large language models (LLMs), through multiple-choice dental specialty entrance examination (DUS) questions. Methods: Dental specialty entrance examination questions related to prosthodontics and oral and dentomaxillofacial radiology up to 2021, excluding visually integrated questions, were prompted into LLMs. Then the LLMs were asked to choose the correct response and specify Bloom’s taxonomy level. After data collection, the LLMs’ ability to recognize Bloom’s taxonomy levels and the correct response rate in different subheadings, the agreement between LLMs on correct and incorrect answers, and the effect of Bloom’s taxonomy level on correct response rates were evaluated. Data were analyzed using McNemar, Chi-square, and Fisher–Freeman–Halton exact tests, and Yate’s continuity correction and Kappa agreement level were calculated (P