Screening/diagnosis of pediatric endocrine disorders through the artificial intelligence model in different language settings

Abstract

This study is aimed at examining the impact of ChatGPT on pediatric endocrine and metabolic conditions, particularly in the areas of screening and diagnosis, in both Chinese and English modes. A 40-question questionnaire covering the four most common pediatric endocrine and metabolic conditions was posed to ChatGPT in both Chinese and English three times each. Six pediatric endocrinologists evaluated the responses. ChatGPT performed better when responding to questions in English, with an unreliable rate of 7.5% compared to 27.5% for Chinese questions, indicating a more consistent response pattern in English. Among the reliable questions, the answers were more comprehensive and satisfactory in the English mode. We also found disparities in ChatGPT’s performance when interacting with different target groups and diseases, with improved performance for questions posed by clinicians in English and better performance for questions related to diabetes and overweight/obesity in Chinese for both clinicians and patients. Language comprehension, providing incomprehensive answers, and errors in key data were the main contributors to the low scores, according to reviewer feedback. Conclusion: Despite these limitations, as ChatGPT continues to evolve and expand its network, it has significant potential as a practical and effective tool for clinical diagnosis and treatment. (Table presented.)