Evaluation of the readability, quality, and accuracy of AI chatbot responses to questions about deleterious oral habits

Abstract

Background: Artificial intelligence (AI)-based chatbots are increasingly used by parents as convenient and fast-access sources of information on health-related topics. This study aimed to assess the readability, accuracy and overall quality of responses provided by ChatGPT-4o, Google Gemini and Microsoft Copilot to questions concerning deleterious oral habits in children. Methods: A total of 43 questions, derived from real-life discussions on the Reddit platform, were revised for clarity and demographic diversity. These were classified into seven categories based on specific types of deleterious oral habits, including thumb sucking, bruxism, pacifier use, bruxism, tongue thrusting, lip sucking, nail biting, and mouth breathing. Responses from each AI chatbot were evaluated using multiple evaluation tools including Flesch Reading Ease (FRE), Flesch-Kincaid Grade Level (FKGL), the modified DISCERN tool (mDISCERN), Global Quality Score (GQS), and misinformation scoring system. Statistical analyses were performed using the Kruskal–Wallis test followed by Dunn’s post hoc test for non-normally distributed variables, and one-way ANOVA with Tukey’s post hoc test for normally distributed variables (p