The expectation maximization algorithm: A short tutorial

Abstract

This tutorial discusses the ExpectationMaximization (EM) algorithm of Demp- ster, Laird and Rubin 1. The approach taken follows that of an unpublished note by Stuart Russel, but fleshes out some of the gory details. In order to ensure that the presentation is reasonably self-contained, some of the results on which the derivation of the algorithm is based are presented prior to the main results. The EM algorithm has become a popular tool in statistical estimation problems involving incomplete data, or in problems which can be posed in a sim- ilar form, such as mixture estimation 3, 4. The EM algorithm has also been used in various motion estimation frameworks 5 and variants of it have been used in multiframe superresolution restoration methods which combine motion estimation along the lines of 2.