Using Impact Evaluation Tools to Unpack the Black Box and Learn What Works

Abstract

Researchers and policy makers are increasingly dissatisfied with the “average treatment effect.” Not only are they interested in learning about the overall causal effects of policy interventions, but they want to know what specifically it is about the intervention that is responsible for any observed effects. This discusses Peck's (2003) approach to creating symmetrically-predicted subgroups for analyzing endogenous features of experimentally evaluated interventions and then it identifies several possible extensions that might help evaluators better understand complex interventions. It aims to enrich evaluation methodologists’ toolbox, to improve our ability to analyze “what works” in addressing important questions for policy and program practice.