A double-loop algorithm to minimize the bethe free energy

Abstract

Recent work (Yedidia, Freeman, Weiss [22]) has shown that stable points of belief propagation (BP) algorithms [12] for graphs with loops correspond to extrema of the Bethe free energy [3]. These BP algorithms have been used to obtain good solutions to problems for which alternative algorithms fail to work [4], [5], [10] [11]. In this paper we introduce a discrete iterative algorithm which we prove is guaranteed to converge to a minimum of the Bethe free energy. We call this the double-loop algorithm because it contains an inner and an outer loop. The algorithm is developed by decomposing the free energy into a convex part and a concave part, see [25], and extends a class of mean field theory algorithms developed by [7],[8] and, in particular, [13]. Moreover, the double-loop algorithm is formally very similar to BP which may help understand when BP converges. In related work [24] we extend this work to the more general Kikuchi approximation [3] which includes the Be the free energy as a special case. It is anticipated that these double-loop algorithms will be useful for solving optimization problems in computer vision and other applications.