Sparse boosting with correlation based penalty

Abstract

In high dimensional setting, componentwise L2 boosting method has been used to construct sparse model of high prediction, but it tends to select many ineffective variables. Several sparse boosting methods, such as, SparseL2 Boosting and Twin Boosting, have been proposed to improve the variable selection of L2boosting algorithm. In this paper, we propose a new general sparse boosting method (GSBoosting). The relations are established between GSBoosting and other well known regularized variable selection methods in orthogonal linear model, such as adaptive Lasso, hard thresholds etc. Simulations results show that GS-Boosting has good performance in both prediction and variable selection. © Springer-Verlag 2012.