Evaluating ChatGPT 3.5 and 4.0 in Oral Medicine and Radiology: A Comparative Query-Based Cross-sectional Study

Abstract

Background: Artificial intelligence (AI) is transforming healthcare, with generative models like ChatGPT offering innovative solutions in education and clinical practice. Objective: This study compares ChatGPT 3.5 and 4.0 in oral medicine and radiology, assessing their accuracy, relevance, and practical utility. Methods: A cross-sectional exploratory study was conducted at the Department of Oral Medicine and Radiology, in a private teaching institution, in Chennai. Fifty open-ended questions (25 each from oral medicine and oral radiology) were posed to both ChatGPT versions. Three blinded experts evaluated responses using a modified 4-point Likert scale. Statistical analysis included intraclass correlation coefficients for interobserver reliability and paired t-tests for mean score comparisons. Results: ChatGPT 4.0 demonstrated significantly higher accuracy and relevance than ChatGPT 3.5, with mean scores of 3.52 vs. 2.96 for oral medicine (P = 0.032) and 3.61 vs. 3.17 for oral radiology (P = 0.010). Strong interobserver agreement confirmed evaluation robustness. ChatGPT 4.0 consistently provided more detailed and clinically relevant responses, highlighting its potential as a supplementary tool in dental education. Conclusion: ChatGPT 4.0 outperforms its predecessor in academic queries related to oral medicine and radiology. These findings emphasize AI’s role in dental education and the need for further research on its clinical applications.