Regression-Discontinuity Analysis

Abstract

Regression discontinuity (RD) methods are one of the standard techniques used in statistics and econometrics to obtain causal inference from observational data. But imple-mentations of RD can have serious problems in practice, especially with the common approach of controlling for high-degree polynomials of the underlying continuous pre-dictor. In a companion paper (Gelman and Imbens, 2014) we present evidence that controlling for high-order polyno-mials in RD analysis results in noisy estimates with poor statistical properties and confidence intervals that are too narrow. In the present paper we discuss evident practical problems with these estimates and how they interact with pathologies of the current system of scientific publication. We demonstrate with a recent well-publicized example in public health where a high-degree polynomial control in an RD analysis led to implausible conclusions. The magni-tude and significance of reported treatment effects were highly sensitive to model specification. We then extend a paper by Green et al. (2009) to illustrate that high-degree polynomial estimates such as those reported in the public health paper are subject to uncertainty and noise not captured by reported p-values. In addition to implying that