Learning from failure: A context-informed perspective on RCTs

Abstract

Discussions of randomised controlled trials (RCTs) in education that do not show an impact regularly focus on the intervention and how it failed to impact on expected measures, with typologies identifying persistent critical points of failure. This paper uses one such RCT—the Integrating English programme—to exemplify the application of a new model to explain failure in RCTs. To do so, the paper develops a set of categories of context drawing on the wider social evaluation field: backdrop, design, operation and interpretation. Thus, the paper exposes critical weak points in the commission and interpretation, as well as the implementation, of an RCT. Our aim is to work towards more robust evaluations by demonstrating that it is not simply the programme design, implementation and evaluation that can contribute to a lack of impact; there can be more fundamental system issues at play.