AI-powered standardised patients: evaluating ChatGPT-4o’s impact on clinical case management in intern physicians

Abstract

Background: Artificial Intelligence is currently being applied in healthcare for diagnosis, decision-making and education. ChatGPT-4o, with its advanced language and problem-solving capabilities, offers an innovative alternative as a virtual standardised patient in clinical training. Intern physicians are expected to develop clinical case management skills such as problem-solving, clinical reasoning and crisis management. In this study, ChatGPT-4o’s served as virtual standardised patient and medical interns as physicians on clinical case management. This study aimed to evaluate intern physicians’ competencies in clinical case management; problem-solving, clinical reasoning, crisis management and explore the impact and potential of ChatGPT-4o as a viable tool for assessing these competencies. Methods: This study used a simultaneous triangulation design, integrating quantitative and qualitative data. Conducted at Aydın Adnan Menderes University, with 21 sixth-year medical students, ChatGPT-4o simulated realistic patient interactions requiring competencies in clinical case management; problem-solving, clinical reasoning, crisis management. Data were gathered through self-assessment survey, semi-structured interviews, observations of the students and ChatGPT-4o during the process. Analyses included Pearson correlation, Chi-square, and Kruskal-Wallis tests, with content analysis conducted on qualitative data using MAXQDA software for coding. Results: According to the findings, observation and self-assessment survey scores of intern physicians’ clinical case management skills were positively correlated. There was a significant gap between participants’ self-assessment and actual performance, indicating discrepancies in self-perceived versus real clinical competence. Participants reported feeling inadequate in their problem-solving and clinical reasoning competencies and experienced time pressure. They were satisfied with the Artificial Intelligence-powered standardised patient process and were willing to continue similar practices. Participants engaged with a uniform patient experience. Although participants were satisfied, the application process was sometimes negatively affected due to disconnection problems and language processing challenges. Conclusions: ChatGPT-4o successfully simulated patient interactions, providing a controlled environment without risking harm to real patients for practicing clinical case management. Although some of the technological challenges limited effectiveness, it was useful, cost-effective and accessible. It is thought that intern physicians will be better supported in acquiring clinical management skills through varied clinical scenarios using this method. Clinical trial number: Not applicable.