Evaluating AI-Generated Language as Models for Strategic Competence in English Language Teaching

Abstract

Strategic competence, the ability to use communication strategies (CS) to overcome challenges and enhance communication effectiveness, is crucial in language learning. However, the coverage of these strategies as well as target models to teach them remain scarce in current instructional materials. This paper represents the first attempt to examine the application of ChatGPT in providing target models of CSs and facilitate L2 learners' strategic competence. ChatGPT-4 was used to generate transcripts of monologues and dialogues around a description task following two types of prompts: with and without a taxonomy of communication strategies (structured and unstructured prompts). Preliminary findings suggest Chat-GPT’s considerable potential in modeling communication strategies. Across the two prompting conditions, the chatbot was able to present a wide range of CSs, including achievement, self-monitoring, time-gaining, and interactive strategies. The highest CS content was found in the structured-prompt dialogue which utilized 9 out of 10 CS sub-types, a more diverse range than typically covered in textbooks, with approximation, circumlocution, and time gaining being most frequently used. In terms of linguistic presentation, the AI-generated transcripts demonstrated appropriate use of CSs, though their linguistic realizations were limited in range. The article concludes with implications for leveraging Chat-GPT to contextualize communication strategies, considerations for prompt engineering, strategy training to proficiency levels, and AI-teacher collaboration.