Experiment’s persistent failure in education inquiry, and why it keeps failing

Abstract

Natural scientists are relaxed about the multiple forms experiment takes in their various fields. Yet in education we have for many years constrained our notion of experiment. This methodological circumscription has been self-imposed on the grounds that experiment of a particular, well-defined form offers the clearest evidence of a link between cause and effect in assessing the impact of interventions. I challenge the legitimacy of this assertion and further argue that the model of intervene-and-experiment is ineffective and misleading. There is currently emerging a large body of findings from such experiment, and evaluations of these—like similar evaluations from a wave of experiments in the 1960s and 1970s across education and the applied social sciences—mainly concur on how disappointing the findings from this kind of work are. I argue that interventions are found to have largely nugatory consequences because the influence of independent variables is routinely overwhelmed by powerful contextual influences. I discuss the significance and nature of these contextual influences and question the legitimacy of the idea that one can test interventions with formal experiment in education. I conclude that the assertion that formal experiment can be fruitfully employed may drive policy in unhelpful directions, as models which may be successful in some circumstances are rejected on the basis of low effectiveness scores, while others in which potential effectiveness is indicated are unproductively imposed where circumstances are unpropitious. I suggest that a more unrestricted interpretation of ‘experiment’ needs to return to education discourse.