A BP-based algorithm for performing Bayesian inference in large perceptron-type networks

Abstract

Although the Bayesian approach provides optimal performance for many inference problems, the computation cost is sometimes impractical. We herein develop a practical algorithm by which to approximate Bayesian inference in large single-layer feed-forward networks (perceptrons) based on belief propagation (BP). Although direct application of BP to the inference problem remains computationally difficult, by introducing methods and concepts from statistical mechanics that are related to the central limit theorem and the law of large numbers, the proposed BP-based algorithm exhibits nearly optimal performance in a practical time scale for ideal large networks. In order to demonstrate the practical significance of the proposed algorithm, an application to a problem that arises in a mobile communications system is also presented. © Springer-Verlag Berlin Heidelberg 2004.