Dialogue sequence detection in movies

Abstract

Dialogue sequences constitute an important part of any movie or television program and their successful detection is an essential step in any movie summarisation/indexing system. The focus of this paper is to detect sequences of dialogue, rather than complete scenes. We argue that these shorter sequences are more desirable as retrieval units than temporally long scenes. This paper combines various audiovisual features that reflect accepted and well know film making conventions using a selection of machine learning techniques in order to detect such sequences. Three systems for detecting dialogue sequences are proposed: one based primarily on audio analysis, one based primarily on visual analysis and one that combines the results of both. The performance of the three systems are compared using a manually marked-up test corpus drawn from a variety of movies of different genres. Results show that high precision and recall can be obtained using low-level features that are automatically extracted. © Springer-Verlag Berlin Heidelberg 2005.