Learning with Compressible Priors

  • Cevher V
  • 117


    Mendeley users who have this article in their library.
  • 52


    Citations of this article.


We describe a set of probability distributions, dubbed compressible priors, whose independent and identically distributed (iid) realizations result in p-compressible signals. A signal x R N is called p-compressible with magnitude R if its sorted coefficients exhibit a power-law decay as x(i) R i d, where the decay rate d is equal to 1/p. p-compressible signals live close to K-sparse signals (K N) in the ℓr-norm (r> p) since their best K-sparse approximation error decreases with O R K 1/r1/p). We show that the membership of generalized Pareto, Students t, log-normal, Fréchet, and log-logistic distributions to the set of compressible priors depends only on the distribution parameters and is independent of N. In contrast, we demonstrate that the membership of the generalized Gaussian distribution (GGD) depends both on the signal dimension and the GGD parameters: the expected decay rate of N-sample iid realizations from the GGD with the shape parameter q is given by 1 q log (N/q). As stylized examples, we show via experiments that the wavelet coefficients of natural images are 1.67-compressible whereas their pixel gradients are 0.95 log (N/0.95)-compressible, on the average. We also leverage the connections between compressible priors and sparse signals to develop new iterative re-weighted sparse signal recovery algorithms that outperform the standard ℓ1-norm minimization. Finally, we describe how to learn the hyperparameters of compressible priors in underdetermined regression problems by exploiting the geometry of their order statistics during signal recovery. 1

Get free article suggestions today

Mendeley saves you time finding and organizing research

Sign up here
Already have an account ?Sign in

Find this document

  • SCOPUS: 2-s2.0-77955672486
  • SGR: 77955672486
  • ISBN: 9781615679119
  • PUI: 364482591


  • Volkan Cevher

Cite this document

Choose a citation style from the tabs below

Save time finding and organizing research with Mendeley

Sign up for free