The Dantzig selector and sparsity oracle inequalities

Abstract

Let where X,X1,..., Xn are i.i.d. random variables in a measurable space (S,A) with distribution Π and ζ,ζ1,. ..,ζn are i.i.d. random variables with Eζ = 0 independent of (X1,...,Xn). Given a dictionary h1..., hN : S → ℝ, let fλ : = Σnj=1 λj h j, λ = (λ1....λN) ∈ ℝN. Given ε > 0, define and in the case where f* : = fλ*, λ* ∈ ℝN, Candes and Tao [Ann. Statist. 35 (2007) 2313-2351] suggested using λ as an estimator of λ*. They called this estimator "the Dantzig selector". We study the properties of fλ as an estimator of f* for regression models with random design, extending some of the results of Candes and Tao (and providing alternative proofs of these results). © 2009 ISI/BS.