A generalization of forward-backward algorithm

Abstract

Structured prediction has become very important in recent years. A simple but notable class of structured prediction is one for sequences, so-called sequential labeling. For sequential labeling, it is often required to take a summation over all the possible output sequences, for instance when estimating the parameters of a probabilistic model. We cannot directly calculate such a summation from its definition in practice. Although the ordinary forward-backward algorithm provides an efficient way to do it, it is applicable to limited types of summations. In this paper, we propose a generalization of the forward-backward algorithm, by which we can calculate much broader types of summations than the conventional forward-backward algorithm. We show that this generalization subsumes some existing calculations required in past studies, and we also discuss further possibilities of this generalization.