Comparing the Quality and Readability of ChatGPT-4-Generated vs. Human-Generated Patient Education Materials for Total Knee Arthroplasty

Abstract

Background The purpose of this study was to evaluate the potential role of artificial intelligence, specifically ChatGPT-4, in generating patient education materials (PEMs) for total knee arthroplasty (TKA). The aim of our study was to compare the quality and readability of PEMs for TKA generated by ChatGPT-4 with those created by human experts to assess the potential for the use of AI in patient education. Materials and methods We assessed the quality and readability of TKA PEMs produced by ChatGPT-4 and five reputable human-generated websites. Readability was compared using Flesch-Kincaid Grade Level and Flesch Reading Ease. The quality of information was compared using the DISCERN criteria. Results ChatGPT-4 PEMs demonstrated a significantly higher reading grade level and lower reading ease score compared to human-generated PEMs (<0.001). Conclusions The utility of ChatGPT-4 for producing TKA PEMs is promising. Notably, the quality and reliability are as good as human-generated resources. However, it is currently limited by readability issues, leading to a recommendation against its use. Future AI enhancements should prioritize readability to ensure information is more accessible. Effective collaboration between AI developers and healthcare professionals is vital for improving patient education outcomes.