Program Impact Estimation with Binary Outcome Variables: Monte Carlo Results for Alternative Estimators and Empirical Examples

Abstract

A frequent challenge in program impact estimation, and causal model- ing more generally, is estimation of the effect of a binary endogenous variable on a binary outcome of interest. We report results from Monte Carlo experiments de- signed to assess the performance of estimators frequently applied in this circum- stance. Many rely on an instrumental variables identification strategy. Even when identification is achieved by functional form, it is widely perceived that instruments generate more credible identification. Our focus is on widely used models avail- able in the popular STATA statistical software package but we also evaluate a semi- parametric instrumental variables random effects model not yet available in STATA. The parameters of interest in these experiments are program impact, test statistics assessing endogeneity and overidentification tests.We consider performance under alternative behavioral circumstances by varying distributional assumptions for un- observables, instrument strength levels, sample sizes, and impact magnitudes. Some models turn in a somewhat disappointing performance. Those that rely on joint nor- mality for identification are not particularly robust to error misspecification, raising questions about whether they should be preferred to the semi-parametric estima- tor (regardless of comparative ease of estimation) or even to simple single equation models that ignore endogeneity. We provide examples of the methods using data from Bangladesh and Tanzania. David