Assessing ChatGPT’s suitability in responding to the public’s inquires on the effects of smoking on oral health

Abstract

Background: With the growing use of Artificial Intelligence (AI) in healthcare, language models like ChatGPT are increasingly being used to provide health information. Given smoking’s significant impact on oral health, including periodontal disease, tooth loss, oral cancer, and impaired healing—this study assesses ChatGPT’s effectiveness in answering inquiries about the effects of smoking on oral health. Methods: A total of 119 smoking-related dental health questions were generated using online tools and categorized into five domains: periodontal conditions, teeth and health, oral hygiene and breath, oral soft tissues, and oral surgery. ChatGPT 3.5 responses were evaluated for usefulness, readability (FKG index and, SMOG index), quality (Global Quality Scale), reliability (CLEAR tool), and understandability/actionability (PEMAT). Statistical analyses included ANOVA, Kruskal-Wallis, Mann-Whitney U, and Spearman correlation tests using SPSS v20. Results: ChatGPT responses were rated as “very useful” (36.1%) or “useful” (42.0%) for most questions. The oral hygiene and breath category received the highest usefulness scores (p =.002). Readability scores indicated that 35.3% of responses were very difficult to read. Surgery-related questions were significantly harder to read (p 70%). Conclusion: ChatGPT provides reliable, high-quality information on smoking’s impact on oral health, suitable for general health education. However, it lacks actionable, personalized guidance, limiting its effectiveness in supporting behavioral change and health management.