Reducing anxiety and enhancing performance: the impact of AI chatbots versus human facilitation on EFL speaking assessment outcomes

Abstract

Introduction: This study investigates the comparative impact of artificial intelligence (AI) chatbots and human facilitation on foreign language speaking anxiety and achievement scores among 48 first-year English Language Teaching students at a Turkish public university. Methods: Using a mixed-methods approach, we administered two comparable speaking examinations—one facilitated by an AI chatbot and the other by human instructors—with Foreign Language Speaking Anxiety Scale assessments preceding each exam. Additionally, students completed a digital literacy assessment to determine technological proficiency levels. Semi-structured interviews provided qualitative insights into student experiences. Results: Results revealed a significant negative correlation between speaking anxiety and achievement scores in human-facilitated examinations (r = −0.500, p < 0.01), but no significant correlation in AI-facilitated examinations (r = −0.042, p > 0.01), suggesting AI’s potential in mitigating anxiety effects. While achievement scores did not significantly differ between test conditions, students with higher digital literacy performed better in AI-facilitated examinations (r = 0.353, p < 0.05). Thematic analysis of qualitative data revealed four major themes: self-perceived speaking proficiency, comparative perceptions of AI versus human evaluation, intentions for future classroom implementation, and feedback quality assessment. Discussion: The findings contribute to the theoretical understanding of technology-mediated assessment in foreign language education and offer practical implications for creating anxiety-reduced assessment environments. Notably, this study extends current research by addressing cross-cultural considerations in AI assessment implementation and examining how technological familiarity influences assessment experiences across diverse educational contexts.