Divergence measures and approximate algorithms for valuation based systems

Abstract

This paper considers an abstract framework for expressing approximate inference algorithms in valuation based systems. It will provide a definition of a ‘more informative’ binary relation between representations of information as well as the basic properties of a divergence measure. The approach is illustrated with the cases of probabilistic reasoning (computation of marginal probabilities and most probable explanation) and with inference problems in propositional logic. Examples of divergence measures satisfying the basic properties will be given for these problems. Finally, we will formulate in an abstract way the mean field variational approach and the iterative belief propagation algorithm.