Preconditioning the Lasso for sign consistency

Abstract

Sign consistency of the Lasso requires the stringent irrepresentable condition. This paper examines whether preconditioning can circumvent this condition. Let X∈ℝ n×p and Y∈ℝ n satisfy the standard linear regression equation. Instead of computing the Lasso with (X, Y), preconditioning first left multiplies by F∈ℝ n×n and then computes the Lasso with (FX, FY). While others have proposed preconditioning for other purposes, we provide the first results that show FX can satisfy the irrepresentable condition even when X fails to satisfy the condition. Preconditioning the Lasso creates a new estimator that is sign consistent in a wider variety of settings. Importantly, left multiplying the regression equation by F does not change β, the vector of unknown coefficients. However, left multiplying this equation by F often inflates the variance of the errors. We propose a class of preconditioners to balance these costs and benefits.