AI-ASSISTED AND TRADITIONAL ASSESSMENT OF THERAPEUTIC COMMUNICATION SKILLS: A QUASI-EXPERIMENTAL STUDY WITH STANDARDIZED PATIENT-BASED EDUCATION

Abstract

Objective: This study aims to examine the effectiveness of standardized patient (SP) based therapeutic communication training and the accuracy, reliability, and feasibility of artificial intelligence (AI) supported assessment in improving nursing students' communication competencies. The premise of the study is to determine to what extent AI can assess complex clinical skills, such as interpersonal communication, in harmony with human raters. Method: This quasi-experimental pre-test/post-test study, including a three-month follow-up measurement, was completed with 64 students (intervention group n=33, control group n=31) after eight of the initially randomized 72 students left the study during the process. A five-session SP-supported therapeutic communication training was applied to the intervention group, while the control group received standard education. Communication performances were scored at baseline, post-training, and follow-up by both human raters and an AI-based assessment system analyzing anonymized transcripts containing coded nonverbal behaviors. Data were analyzed using mixed-design ANOVA, independent samples t-tests, Pearson correlations, and intraclass correlation coefficients (ICC). Results: Students in the intervention group showed significant improvement in therapeutic communication skills compared to the control group at all measurement times (p