Control variates for estimation based on reversible Markov chain Monte Carlo samplers

Abstract

A general methodology is introduced for the construction and effective application of control variates to estimation problems involving data from reversible Markov chain Monte Carlo samplers. We propose the use of a specific class of functions as control variates, and we introduce a new consistent estimator for the values of the coefficients of the optimal linear combination of these functions. For a specific Markov chain Monte Carlo scenario, the form and proposed construction of the control variates is shown to provide an exact solution of the associated Poisson equation. This implies that the estimation variance in this case (in the central limit theorem regime) is exactly zero. The new estimator is derived from a novel, finite dimensional, explicit representation for the optimal coefficients. The resulting variance reduction methodology is primarily (though certainly not exclusively) applicable when the simulated data are generated by a random-scan Gibbs sampler. Markov chain Monte Carlo examples of Bayesian inference problems demonstrate that the corresponding reduction in the estimation variance is significant, and that in some cases it can be quite dramatic. Extensions of this methodology are discussed and simulation examples are presented illustrating the utility of the methods proposed. All methodological and asymptotic arguments are rigorously justified under essentially minimal conditions. © 2011 Royal Statistical Society.