Generalizability of Written Expression Curriculum-Based-Measurement in the German Language: What Are the Major Sources of Variability?

Abstract

This study aimed to identify the sources of measurement error that contribute to the intraindividual variability of written expression curriculum-based measurement (CBM-W) and assess how many German writing samples of 3 or 5 min duration are necessary to make sufficiently reliable relative and absolute decisions. Students in grade 3 (N = 128) and grade 6 (N = 118) wrote five CBM-W probes of 5 min each within 1 week, which were scored for commonly used metrics (i.e., words written, correct writing sequences). Analyses within the generalizability theory framework showed that between-student differences accounted for 36–60% of the variance. The student × writing prompt interaction was the largest source of variability, particularly among younger students (44%), while writing prompt per se and writing time explained no variance. Two to four writing samples of 3 min are sufficient for most scoring methods to achieve relative reliability >0.80. CBM-W in German proved inadequate for the grade levels studied for absolute decisions. These findings imply that CBM-W in this form in German-speaking primary grades is suitable as a universal screening tool but not as a tool for progress monitoring of individual students.