Studying the Effectiveness of Programs and Initiatives in Higher Education Using the Regression-Discontinuity Design

Abstract

The goal behind any successful program evaluation is to be able to compare a treatment group to a control group where both of these groups are equivalent in all respects except for the group assignment. In other words, if individuals in a treatment group are equivalent in all respects to individuals in a control group, and if the treatment proves to be effective, then this would suggest that the effect is due to the impact of the treatment program and not to differences between the those participants in the treatment group and those participants in the control group. Without establishing treatment and control groups that are equivalent in all respects, no claim can be made as to whether or not it is the treatment program itself that caused the effect. This chapter describes how the regression-discontinuity design can be used to make causal inferences when it may not be possible to establish treatment and control groups using a true random assignment. I begin by providing some details about cause-andeffect relationships. I then describe the general theory that guides the regression-discontinuity design and elaborate on the use of an assignment variable. I also address issues such as model specification, sample size considerations, including additional control variables, and selection bias. I then address some problems with the regression-discontinuity design which could bias the estimate of the treatment effect such as functional form specification, crossovers, program attrition, and other potential outside effects. The regression-discontinuity design is then illustrated in detail. Finally, I provide a summary of what researchers need to consider before using the regression-discontinuity design.