Statistical variable selection and causality in the social and behavioral sciences

Abstract

The problem of “variable selection” is a fundamental one across the sciences. In its broadest terms, this problem would be at least part of the general issue of theory selection and comparison. However, there is a more circumscribed problem that concerns primarily the choice of variables for the best fitting model, given some set of data, usually observational in nature, and specific statistical techniques, typically multiple regression. There is a deep strand in econometrics and other applied social, behavioral, and biomedical science statistics to want formal decision rules or algorithms to pick out variables. The paper examines seven such formal procedures using a simulated data set with known causal relations. The conclusion is that seven often-used procedures make systematic causal errors. Some suggestions about better alternatives conclude.