Very accurate posterior approximations based on finite mixtures of the hyperparameters conditionals

Abstract

Consider a posterior density π (λ, φ) such that both π (λ | φ) and π (φ | λ) are known. We propose to approximate π (λ, φ) by π (λ | φ) over(π, ^) (φ), where over(π, ^) (φ) is a finite mixture of the posterior conditionals π (φ | λ). The weights and components of the mixture are chosen to minimize an approximate f-divergence between the approximate and the actual posterior. These approximate divergences are computed through an importance sampling idea using a simulated sample from the same finite mixture approximations. For the special case of the χ2 or Harmonic divergences, once the minimum approximate divergences have been obtained, they can be plugged into total variation type inequalities to obtain precision limits for the corresponding approximations of posterior expectations of interest. When the algorithm can be used-namely, when both full conditionals π (λ | φ) and π (φ | λ) are known, it requires little computational, programming and diagnosing effort. Moreover, we present several examples which show that the approximations produced are extremely accurate, even when a small number of components are included in the mixture approximation. © 2005 Elsevier B.V. All rights reserved.