From Stein's Unbiased Risk Estimates to the Method of Generalized Cross Validation

Abstract

This paper concerns the method of generalized cross validation (GCV), a promising way of choosing between linear estimates. Based on Stein estimates and the associated unbiased risk estimates (Stein, 1981), a new approach to GCV is developed. Many consistency results are obtained for the cross-validated (Steinized) estimates in the contexts of nearest-neighbor nonparametric regression, model selection, ridge regression, and smoothing splines. Moreover, the associated Stein's unbiased risk estimate is shown to be uniformly consistent in assessing the true loss (not the risk). Consistency properties are examined as well when the sampling error is unknown. Finally, we propose a variant of GCV to handle the case that the dimension of the raw data is known to be greater than that of their expected values.