Consistent tuning parameter selection in high-dimensional group-penalized regression

Abstract

Various forms of penalized estimators with good statistical and computational properties have been proposed for variable selection respecting the grouping structure in the variables. The attractive properties of these shrinkage and selection estimators, however, depend critically on the choice of the tuning parameter. One method for choosing the tuning parameter is via information criteria, such as the Bayesian information criterion (BIC). In this paper, we consider the problem of consistent tuning parameter selection in high dimensional generalized linear regression with grouping structures. We extend the results of the extended regularized information criterion (ERIC) to group selection methods involving concave penalties and then investigate the selection consistency with diverging variables in each group. Moreover, we show that the ERIC-type selector enables consistent identification of the true model and that the resulting estimator possesses the oracle property even when the number of group is much larger than the sample size. Simulations show that the ERIC-type selector can significantly outperform the BIC and cross-validation selectors when choosing true grouped variables, and an empirical example is given to illustrate its use.