Learning Algorithm for Boltzmann Machines Using Max-Product Algorithm and Pseudo-Likelihood

Abstract

Boltzmann machines are parametric probabilistic models for the statistical machine learning, forming Markov random fields. Owing to their normalization constant, inference and learning in Boltzmann machines are generally classified under NP-hard problems. Maximum pseudo-likelihood estimation is an effective approximate learning method for Boltzmann machines. However, in principle, we cannot use this method for incomplete data sets, except for some special cases. In this paper, we propose a new learning algorithm for Boltzmann machines with incomplete data sets by generating a pseudo-complete data set from a given incomplete data using the max-product algorithm and the Markov chain Monte Carlo method, and then, by applying maximum pseudo-likelihood estimation to the pseudo-complete data set.