Causal interaction and external validity: Obstacles to the policy relevance of randomized evaluations

Abstract

The ability to generalize effects estimated from randomized experiments is critical for their relevance to policy. Framing that problem in terms of causal interaction reveals the extent to which the literature to date has failed to adequately address external validity. An analogy with matching estimators illustrates the current inconsistency in approaches to estimating causal relationships and generalizing these estimates to other populations and contexts. Contrary to some claims, atheoretic replication is not a plausible solution. Better knowledge of, and more information on, interacting factors is required for credible, formal extrapolation. In the absence of that, modesty is recommended.