On the Effectiveness of Pop-Up English Language Glossary Accommodations for EL Students in Large-Scale Assessments

Abstract

This study examined the effectiveness and influence on validity of a computer-based pop-up English glossary accommodation for English learners (ELs) in grades 3 and 7. In a randomized controlled trial, we administered pop-up English glossaries with audio to students taking a statewide accountability English language arts (ELA) and mathematics assessments. As is typically found, EL students exhibited lower achievement scores than non-EL students in all portions of the test. The pop-up glossaries provided inconsistent benefit for EL students. There was some evidence that the pop-up English glossaries had a minimal inhibitory effect for 3rd-grade students on both the ELA and mathematics assessment. Furthermore, 7th-grade ELs also showed slightly inhibited performance when using the pop-up glossary on the mathematics assessment. However, 7th-grade EL students had a performance benefit when using the pop-up glossary on the ELA assessment. We discuss how increased cognitive load placed on younger students may play a role in diminishing performance when using pop-up glossaries. We explore potential explanations for the difference outcomes between mathematics and ELA in grade 7.