Hidden Markov Models and the Baum-Welch Algorithm

Abstract

The lectures of previous Shannon Lecturers fall into several categories such as introducing new areas of research, resuscitating areas of research, surveying areas identified with the lecturer, or reminiscing on the career of the lecturer. In this talk I decided to restrict the subject to the Baum-Welch algorithm and some of the ideas that led to its development. I am sure that most of you are familiar with Markov chains and Markov processes. They are natural models for various communication channels in which channel conditions change with time. In many cases it is not the state sequence of the model which is observed but the effects of the process on a signal. That is, the states are not observable but some functions, possibly random, of the states are observed. In some cases it is easy to assign the values of the parameters to model a channel. All that remains is to determine what probabilities are desired and derive the necessary algorithms to compute them. In other cases, the choice of parameter values is only an estimate and it is desired to find the best values. The usual criterion is maximum likelihood. That is: find the values of parameters which maximizes the probability of the observed data. This is the problem that the Baum-Welch computation addresses.