Using ChatGPT for Kidney Transplantation: Perceived Information Quality by Race and Education Levels

Abstract

Background: Kidney transplantation is a complex process requiring extensive preparation and ongoing monitoring. Artificial intelligence (AI)-powered chatbots hold potential for providing accessible health information, but our understanding of their role in offering health advice for kidney transplantation and how individuals assess such advice remains limited. This study investigates how individuals evaluate ChatGPT's responses to kidney transplantation questions in terms of information quality and empathy, focusing on potential differences across race/ethnicity and educational backgrounds. Methods: We collected Reddit posts (N = 4624) regarding kidney transplantation and selected 86 questions to represent typical clinician inquiries. These questions were used as input prompts for ChatGPT. A total of 565 participants assessed ChatGPT's responses through online surveys, rating information quality and empathy using Likert scales. Results: Multilevel analyses (N = 2825) show that there is a significant interaction between race/ethnicity and education levels in various measures related to perceived information quality, but not perceived empathy of ChatGPT's responses: accuracy (p < 0.05); authenticity (p < 0.01); believability (p < 0.05); informativeness (p = 0.053); usefulness (p < 0.05); recognizing users’ feelings (p = 0.70) and understanding feelings and situations (p = 0.65). Among non-White individuals, higher education levels predicted higher perceived quality of ChatGPT's responses across all information quality measures. Notably, this trend was reversed for White individuals, where higher education levels led to lower perceived information quality. Conclusions: Our results highlight the importance of developing AI tools sensitive to diverse communication styles and information needs.