Cognitive diagnostic assessment of EFL learners’ listening barriers through incorrect responses

Abstract

English as Foreign Language (EFL) learners’ cognitive processes have been a research focus in listening assessment. Most studies use correct responses as data, but undervalue the rich information of the incorrect answers or options (in the case of multiple choice questions, MCQ). However, the MCQ distractors are often intentionally designed to reveal learners’ problems or barriers. In order to diagnose the EFL learners’ listening barriers through incorrect responses, Cognitive Diagnostic Models (CDMs) for bugs were adopted, hence the name Bug-CDMs. First, five EFL listening barrier attributes were identified and two Bug Q-matrices were developed to comparatively analyze the learner’s responses with different Bug-CDMs. The results revealed that Bug-GDINA was the optimal model, and the most prevalent barriers were semantic understanding and vocabulary recognition. These barriers confirmed both compensatory and non-compensatory relationships in causing listening comprehension failures. The study proved the feasibility of Bug-GDINA in diagnosing listening barriers from the incorrect responses. Limitations and suggestions for further research were also proposed.