Should we search for a global minimizer of least squares regularized with an ℓ 0 penalty to get the exact solution of an under determined linear system?

Abstract

We study objectives ℱ d combining a quadratic data-fidelity and an ℓ 0 regularization. Data d are generated using a full-rank M × N matrix A with N>M. Our main results are listed below. Minimizers û of ℱ d are strict if and only if length(support(û)) ≤ M and the submatrix of A whose columns are indexed by support(û) is full rank. Their continuity in data is derived. Global minimizers are always strict. We adopt a weak assumption on A and show that it holds with probability one. Data read d = Aü where length(support(ü))≤ M - 1 and the submatrix whose columns are indexed by support(ü) is full rank. Among all strict (local) minimizers of ℱ d with support shorter than M-1, the exact solution û = ü is the unique vector that cancels the residual. The claim is independent of the regularization parameter. This û = ü is usually a strict local minimizer where ℱ d does not reach its global minimum. Global minimization of ℱ d can then prevent the recovery of ü. A numerical example (A is 5 x 10) illustrates our main results. © 2012 Springer-Verlag.