Learning out of leaders

Abstract

The paper investigates the estimation problem in a regression-type model. To be able to deal with potential high dimensions, we provide a procedure called LOL-for learning out of leaders-with no optimization step. LOL is an autodriven algorithm with two thresholding steps. A first adaptive thresholding helps to select leaders among the initial regressors to obtain a first reduction of dimensionality. Then a second thresholding is performed on the linear regression on the leaders. The consistency of the procedure is investigated. Exponential bounds are obtained, leading to minimax and adaptive results for a wide class of sparse parameters, with (quasi) no restriction on the number p of possible regressors. An extensive computational experiment is conducted to emphasize the practical good performances of LOL. © 2012 Royal Statistical Society.