Exact Bayesian inference via data augmentation

Abstract

Data augmentation is a common tool in Bayesian statistics, especially in the application of MCMC. Data augmentation is used where direct computation of the posterior density, π(θ|x), of the parameters θ, given the observed data x, is not possible. We show that for a range of problems, it is possible to augment the data by y, such that, π(θ|x,y) is known, and π(y|x) can easily be computed. In particular, π(y|x) is obtained by collapsingπ(y,θ|x) through integrating out θ. This allows the exact computation of π(θ|x) as a mixture distribution without recourse to approximating methods such as MCMC. Useful byproducts of the exact posterior distribution are the marginal likelihood of the model and the exact predictive distribution.