Explaining Variation in Treatment Effects: An Impact Evaluation and Mixed-Methods Study of Variation in Early Grade Reading Program Effects

Abstract

This article contributes to understanding the effectiveness of early grade reading interventions in low-income-country contexts and how and why such interventions vary in their effectiveness across schools. It presents the results from an impact evaluation of an early grade reading intervention in Haiti with a mixed-methods analysis of factors that explain variation in program effects. We study program impact using a randomized controlled trial experimental research design and find significant treatment impact on students’ literacy skills, including on students’ reading comprehension in Creole. We use a mixed-methods design to explore variation in program effects across school sites. Key factors that explain variation in program effects include the frequency of instructional coaching visits; the dosage and uptake of treatment, which was linked to student and teacher absenteeism and the exclusive use of the treatment curriculum; the behaviors of school leaders; the fluency and quality of lesson delivery; and the remoteness of schools.