Deterministic annealing variant of variational Bayes method

Abstract

The Variational Bayes (VB) method is widely used as an approximation of the Bayesian method. Because the VB method is a gradient algorithm, it is often trapped by poor local optimal solutions. We introduce deterministic annealing to the VB method to overcome such a local optimal problem. A temperature parameter is introduced to the free energy for controlling the annealing process deterministically. Applying the method to a mixture of Gaussian models and hidden Markov models, we show that it can obtain the global optimum of the free energy and discover optimal model structure. © 2008 IOP Publishing Ltd.