Differences in Reading Screening Accuracy by Percentile Cutoff and English Proficiency: Feature Selection and Group-Wise Prediction Evaluation

Abstract

This exploratory predictive study investigates our capacity to equitably predict the risk of developing reading difficulties based on a battery of tasks tapping into commonly used reading screening constructs. A key question pertains to the extent to which we can make predictions with equal accuracy for native English speakers (EO) and emerging multilinguals (English learners [ELs] with sufficient English proficiency to participate in English screening). This prediction exercise takes place in a linguistically diverse sample, over different prediction intervals (end of the same and subsequent academic year), and across a range of percentile cutoffs (1st–25th) defining reading difficulty. Specifically, using a sample of 1692 kindergarteners and first graders from 24 schools across 13 Californian public school districts, we (a) identified the most important reading screening tasks for a range of percentile cutoffs defining “risk” using random forest feature selection; (b) built a series of logistic regression classifiers; and (c) evaluated these models' performances for ELs and EO students separately. When risk is defined as performance below the tenth percentile or higher, the same tasks generally surface as the most important predictors of later word reading difficulty (letter naming and sounds for kindergarten; word and non-word reading for first graders), though with different relative importance for EO versus EL students and for the different prediction intervals. Phonological skills (blending, deletion) also show to be more predictive for ELs. Though most models can be optimized to exceed sensitivity and specificity values of 0.80, classification accuracy is higher for EO students, especially for 2-year predictions. Though models meeting conventionally used minimum accuracy requirements for both the EO and EL groups can be produced, more work is needed to achieve truly linguistically equitable (i.e., equally accurate) screening. We discuss the equity implications and feasibility of universal reading screening using English screening tasks in linguistically diverse populations.