Investigating the Accuracy of ChatGPT as a Writing Error Correction Tool

Abstract

Artificial intelligence revolution becomes a trend in most aspects of life. ChatGPT, an AI chatbot, has impacted various domains, including education and language learning. Enhancing writing abilities of ESL learners requires frequent writing practice and feedback, which ChatGPT can easily provide. However, ChatGPT’s accuracy in identifying and correcting ESL writing errors remains unknown. This study explores writing errors detected by ChatGPT, and those beyond its capabilities, through content analysis that combines quantitative and qualitative methods, utilizing pre-created data sets and its computational processing. The findings revealed that ChatGPT demonstrated high-performance accuracy for eight categories, encompassing verb forms, word order, passive voice, conditional sentences, subject-verb agreement, singular/plural nouns, word form, and word choices. Its performance was moderate in three categories—articles, modals, and verb tense—while it showed low performance in four categories, including unclear messages, connectors, sentence structure, and non-idiomatic expressions.