On theoretical justification of the forward–backward algorithm for the variational learning of Bayesian hidden Markov models

Abstract

In the variational learning process of a Bayesian Hidden Markov model, the forward-backward algorithm is heuristically applied without theoretical justification. This is potentially problematic, because the original derivation of the forward-backward algorithm implicitly requires the parameters to be normalized, which does not hold in the variational learning process of Bayesian HMM. In this paper, we prove that such a requirement is not necessary for the forward-backward algorithm to obtain the correct result. We prove the result from two perspectives. The first proof straightforwardly verifies that implementing the forward-backward algorithm with the unnormalised parameters is equivalent to implementing it with the normalized parameters. The second proof provides a new derivation of the forward-backward algorithm without hidden Markov assumptions and probabilistic meanings of the parameters. As a result, we justify that applying the forward-backward algorithm is theoretically correct and reasonable in the variational learning of Bayesian hidden Markov models.