Theoretical analysis of cross-validation(CV)-EM algorithm

Abstract

Expectation-Maximization (EM) algorithm is a typical method to estimate parameters of a model with hidden variables and is widely used for many applications. The EM algorithm is simple but sometimes overfits to specific examples and its likelihood diverges to infinite. To overcome the problem of overfitting, Shinozaki and Osterndorf have proposed the CV-EM algorithm in which the cross-validation technique is incorporated into the conventional EM algorithm, and have demonstrated validity of the algorithm with numerical experiments. In this article, we theoretically investigate properties of the CV-EM algorithm with an asymptotic analysis and reveal its mechanism of robustness. © 2010 Springer-Verlag Berlin Heidelberg.