Optimal Combination of Linear and Spectral Estimators for Generalized Linear Models

Abstract

We study the problem of recovering an unknown signal x given measurements obtained from a generalized linear model with a Gaussian sensing matrix. Two popular solutions are based on a linear estimator x^ L and a spectral estimator x^ s. The former is a data-dependent linear combination of the columns of the measurement matrix, and its analysis is quite simple. The latter is the principal eigenvector of a data-dependent matrix, and a recent line of work has studied its performance. In this paper, we show how to optimally combine x^ L and x^ s. At the heart of our analysis is the exact characterization of the empirical joint distribution of (x, x^ L, x^ s) in the high-dimensional limit. This allows us to compute the Bayes-optimal combination of x^ L and x^ s, given the limiting distribution of the signal x. When the distribution of the signal is Gaussian, then the Bayes-optimal combination has the form θx^ L+ x^ s and we derive the optimal combination coefficient. In order to establish the limiting distribution of (x, x^ L, x^ s) , we design and analyze an approximate message passing algorithm whose iterates give x^ L and approach x^ s. Numerical simulations demonstrate the improvement of the proposed combination with respect to the two methods considered separately.