On the use of evidence and external validity in the evaluation of social interventions: A critical overview

Abstract

Objective/Context: There is a growing criticism of the mainstream evaluations of the social interventions (e.g. RCTs) which shape evidence-based policies. This article focuses on two central aspects of this question: I) the notion of evidence, as a concept and a result of a process that entails reasoning, and ii) the understanding of external validity. Methodology: Through an analysis of the literature on evaluation and the philosophy of knowledge, I deconstruct the concepts of causality in the social sciences. This exercise allows me to distinguish between successionist and generative theories of causality and establish criteria to critically examine some epistemological postulates of experimental evaluation techniques. Conclusions: By themselves, experimental methods of evaluation do not allow for informed decisions about an efficient investment of resources. Despite their strength in quantifying possible causal effects, counterfactual statistical analyses need to be complemented by forms of qualitative reasoning in order to answer questions about the direct and indirect causes of the results of social interventions, and thus strengthen our understanding of the extrapolation of social policies or programs from certain contexts. Contribution: There are few studies in Spanish which criticize experimental evaluation techniques or suggest alternatives to them. Instead of summarizing the arguments of other authors, this article presents a coherent narrative which asks us to rethink the current role of the evaluation of social interventions.