Automated Person Identification in Video

Abstract

We describe progress in the automatic detection and identification of humans in video, given a minimal number of labelled faces as training data. This is an extremely challenging problem due to the many sources of variation in a person's imaged appearance: pose variation, scale, illumination, expression, partial occlusion, motion blur, etc. The method we have developed combines approaches from computer vision, for detection and pose estimation, with those from machine learning for classification. We show that the identity of a target face can be determined by first proposing faces with similar pose, and then classifying the target face as one of the proposed faces or not. Faces at poses differing from those of the training data are rendered using a coarse 3-D model with multiple texture maps. Furthermore, the texture maps of the model can be automatically updated as new poses and expressions are detected. We demonstrate results of detecting three characters in a TV situation comedy. © Springer-Verlag 2004.