Shot Boundary Detection for Automatic Video Analysis of Historical Films

Abstract

In automatic video content analysis and film preservation, Shot Boundary Detection (SBD) is a fundamental pre-processing step. While previous research focuses on detecting Abrupt Transitions (AT) as well as Gradual Transitions (GT) in different video genres such as sports movies or news clips only few studies investigate in the detection of shot transitions in historical footage. The main aim of this paper is to create an SBD mechanism inspired by state-of-the-art algorithms which is applied and evaluated on a self-generated historical dataset as well as a publicly available dataset called Clipshots. Therefore, a three-stage pipeline is introduced consisting of a Candidate Frame Range Selection based on the network DeepSBD, Extraction of Convolutional Neural Network (CNN) Features and Similarity Calculation. A combination of pre-trained backbone CNNs such as ResNet, VGG19 and SqueezeNet with different similarity metrics like Cosine Similarity and Euclidean Distance are used and evaluated. The outcome of this paper displays that the proposed algorithm reaches promising results on detecting ATs in historical videos without the need of complex optimization and re-training processes. Furthermore, it points out the main challenges concerning historical footage such as damaged film reels, scratches or splices. The results of this paper contribute a significant base for future research on automatic video analysis of historical videos.