Comparative Evaluation of Artificial Intelligence Chatbots in Delivering Palliative Care Education to Intensive Care Unit Caregivers— A Cross-platform Analysis: A Brief Communication

Abstract

Background and aims: Caregivers of intensive care unit (ICU) patients with advanced chronic illness face significant psychological stress and information gaps regarding palliative care. Generative artificial intelligence (AI) tools like ChatGPT and Google Gemini may offer scalable, personalized educational support. This study aimed to compare the quality of academic content generated by these AI chatbots in terms of readability, sentiment, understandability, actionability, and expert-rated accuracy and completeness. Materials and methods: On December 10, 2024, ChatGPT and Google Gemini (free browser versions) were queried with a standardized prompt on ICU palliative care for caregivers. Outputs were evaluated for readability (eight validated indices), sentiment polarity (online tool), understandability and actionability, patient education materials assessment tool for printable materials (PEMAT-P), and expert panel (n = 7) assessments of accuracy and completeness using a 5-point Likert scale. Statistical comparisons were conducted using paired t-tests. Results: ChatGPT and Gemini produced content of comparable readability (12.96 vs 13.21). Sentiment scores favored ChatGPT (+7.8 vs –39.1). Both chatbots achieved similar scores in understandability and actionability (91.7 and 80%, respectively). Accuracy scores were similar (p = 0.36), while completeness was significantly higher for ChatGPT (95% CI 0.08–1.07; p = 0.025). Conclusion: Both AI tools generated content suitable for caregiver education in ICUs. ChatGPT demonstrated more neutral sentiment and greater completeness, supporting a potential role in critical care communication and education of caregivers in the ICU.