Least angle regression and LASSO for large datasets

Abstract

Least angle regression and LASSO (l1-penalized regression) offer a number of advantages in variable selection applications over procedures such as stepwise or ridge regression, including prediction accuracy, stability, and interpretability. We discuss formulations of these algorithms that extend to datasets in which the number of observations could be so large that it would not be possible to access the matrix of predictors as a unit in computations. Our methods require a single pass through the data for orthogonal transformation, effectively reducing the dimension of the computations required to obtain the regression coefficients and residual sum of squares to the number of predictors, rather than the number of observations. © 2009 Wiley Periodicals, Inc.