High level video temporal segmentation

Abstract

In this paper we propose a novel and complete video structuring/ segmentation framework, which includes shot boundary detection, keyframe selection and high level clustering of shots into scenes. In a first stage, an enhanced shot boundary detection algorithm is proposed. The approach extends the state-of-the-art graph partition model and exploits a scale space filtering of the similarity signal which makes it possible to significantly increase the detection efficiency, with gains of 7,4% to 9,8% in terms of both precision and recall rates. Moreover, in order to reduce the computational complexity, a two-pass analysis is performed. For each detected shot we propose a leap keyframe extraction method that generates static summaries. Finally, the detected keyframes feed a novel shot clustering algorithm which integrates a set of temporal constraints. Video scenes are obtained with average precision and recall rates of 85%. © 2011 Springer-Verlag.