An Exploratory Reliability and Content Analysis of the CEFR-Japan’s A-Level Can-Do Statements

Abstract

Both the Common European Framework of Reference (CEFR) and the CEFR-Japan (CEFR-J), an alternate version designed for Japanese learners of English, provide measurements of language proficiency via assessment or self-assessment on scales of descriptors of communicative competences (known as can-do statements). Al-though extensive empirical evidence supports these claims for the CEFR, the same cannot yet be said of the CEFR-J. Mokken scaling was thus used to measure the reliability of can-do statement scales from the five skills of the CEFR-J’s five A sublevels of A1.1, A1.2, A1.3, A2.1, and A2.2. Statements that negatively affected the reliability of the scale were analysed. Lower reliability was attributed to characteristics spe-cific to participants (homogeneity of the population, familiarity with the task, and if the material was recently studied), and content of the statement itself (whether it implied more than one language skill or none at all, whether it contained a contradic-tion, or was confusing or unfamiliar). Modifications to increase the reliability of can-do statement scales and limitations of using illustrative descriptor-based systems as measurement instruments are discussed.