Mining TV Broadcasts 24/7 for recurring video sequences

Abstract

Monitoring and analyzing TV broadcasts is an important task in the media as well as the advertising business. An important subtask is the frame-accurate detection of recurring video sequences. Examples of recurring video sequences are commercials, channel advertisements, channel intros, and newscast intros. Most of these different kinds of repeating video clips can automatically be classified by further analyzing their temporal and visual properties. In this work we introduce an algorithm and a real-time system for recognizing recurring video sequences frameaccurately in a highly effective and efficient manner. The algorithm does not require any temporal pre-segmentation by shot detection and can thus, in principle, be applied to any kind of temporal signal. It is frame-accurate, meaning that it exactly identifies with which frame a repeating sequence starts and ends. Thus, the temporal accuracy is 40 milliseconds for PAL and 33 milliseconds for NTSC videos. On a standard PC desktop a 24-hour live-stream can be processed in about 4 hours including the computational expensive video decoding. To achieve this efficiency the algorithmexploits an inverted index for identifying similar frames rapidly.Gradientbased image features are mapped to the index by means of a hash function. The search algorithm consists of two steps: firstly searching for recurring short segments of 1 second duration and secondly assembling these small segments into the set of repeated video clips. In our experiments we investigate the sensitivity of the algorithm concerning all system parameters and apply it to the detection of unknown commercials within 24 and 48 hours of various TV channels. It is shown that the method is an excellent technique for searching for unknown commercials. Currently the system is used 24 hours a day, 7 days a week in various countries to log all commercials broadcast without manual intervention. © 2010 Springer-Verlag Berlin Heidelberg.