High patient and surgeon satisfaction with ChatGPT-generated responses to real patient questions regarding total knee arthroplasty

Abstract

Background: Chat-Generative Pretrained Transformer (ChatGPT) has proven its value in medical information acquisition and perioperative education. This study aimed to evaluate correlated factors to explore the value of ChatGPT in patient communication. Methods: Perioperative questions were collected from 44 consecutive total knee arthroplasty (TKA) patients (mean age: 68.5 ± 8.2 years; 65% female; mean education: 4.8th grade) at a tertiary hospital between January and March 2024. Each patient provided five questions, for a total of 220 questions. The selected questions were answered by ChatGPT-3.5. Questionnaires were created and sent to 81 surgeons across hospital tiers (III-A, II-A, III-B) and 53 perioperative TKA patients who were not among the patients who provided the original questions. Some features of the respondents were collected and investigated for analysis. Readability scores were assessed via the Flesch‒Kincaid Grade Level (FKGL). Results: The most common questions concerned operation costs and payments (27 patients), postoperative pain (25 patients), and the procedure details and implants used (19 patients). The mean satisfaction rating of the 81 surgeons regarding the ChatGPT responses was 4.72/5; among the 53 patients, the mean satisfaction rating was 4.99/5. Professionalism and accuracy received scores of 4.41/5 and 4.36/5, respectively. More than half of the surgeons (55.6%) reported a willingness to use ChatGPT to answer nearly all (80%-100%) of their patients' questions. Correlation analysis revealed that a surgeon’s score was influenced by the level of the hospital in which they worked and their surgical experience. The FKGL readability scores of the ChatGPT responses were high. Conclusions: Patients' responses to the questionnaires revealed their real clinical needs. The results of the study showed that ChatGPT has the potential to be a tool for communicating with patients. Understanding the responses generated by ChatGPT requires a high level of education, although the minimum level of education required can be reduced.