Gaussian approximations for probability measures on Rd

Abstract

This paper concerns the approximation of probability measures on Rd with respect to the Kullback-Leibler divergence. Given an admissible target measure, we show the existence of the best approximation, with respect to this divergence, from certain sets of Gaussian measures and Gaussian mixtures. The asymptotic behavior of such best approximations is then studied in the small parameter limit where the measure concentrates; this asympotic behavior is characterized us- ing convergence. The theory developed is then applied to understand the frequentist consistency of Bayesian inverse problems in finite dimensions. For a fixed realization of additive observational noise, we show the asymptotic normality of the posterior measure in the small noise limit. Tak- ing into account the randomness of the noise, we prove a Bernstein-Von Mises type result for the posterior measure.