Can Natural Language Processing (NLP) Provide Consultancy to Patients About Edentulism Teeth Treatment?

Abstract

Aim This study aimed to evaluate the accuracy and quality of the answers given by artificial intelligence (AI) applications to the questions directed at tooth deficiency treatments. Materials and methods Fifteen questions asked by patients/ordinary people about missing tooth treatment were selected from the Quora platform. Questions were asked to the ChatGPT-4 (OpenAI Inc., San Francisco, California, United States) and Copilot (Microsoft Corporation, Redmond, Washington, United States) models. Responses were assessed by two expert physicians using a five-point Likert scale (LS) for accuracy and the Global Quality Scale (GQS) for quality. To assess the internal consistency and inter-rater agreement of ChatGPT-4 and Copilot, Cronbach's alpha, Spearman-Brown's coefficient, and Guttman's split-half coefficient were calculated to measure the reliability and internal consistency of both instruments (alpha=0.05). Results Copilot showed a mean LS value of 3.83+/-0.36 and ChatGPT-4 showed a lower mean value of 3.93+/-0.32. ChatGPT-4's GQS mean value (3.9+/-0.28) is also higher than Copilot (3.83+/-0.06) (p<0.001). Conclusion It can be said that AI chatbots gave highly accurate and consistent answers to questions about the treatment of toothlessness. With the ever-developing technology, AI chatbots can be used as consultants for dental treatments in the future. Copyright © 2024, Guzelce Sultanoglu et al.