Model averaging, asymptotic risk, and regressor groups

Abstract

This paper examines the asymptotic risk of nested least-squares averaging esti-mators when the averaging weights are selected to minimize a penalized least-squares criterion. We find conditions under which the asymptotic risk of the averaging estimator is globally smaller than the unrestricted least-squares esti-mator. For the Mallows averaging estimator under homoskedastic errors, the con-dition takes the simple form that the regressors have been grouped into sets of four or larger. This condition is a direct extension of the classic theory of James– Stein shrinkage. This discovery suggests the practical rule that implementation of averaging estimators be restricted to models in which the regressors have been grouped in this manner. Our simulations show that this new recommendation re-sults in substantial reduction in mean-squared error relative to averaging over all nested submodels. We illustrate the method with an application to the regression estimates of Fryer and Levitt (2013).