Adaptive video summarization via robust representation and structured sparsity

Abstract

To improve faster browsing and more efficient content indexing of huge video collections, video summarization has emerged as an important area of research for the multimedia community. One of the mechanisms to generate video summaries is to extract keyframes which represent the most important content of the video. However, there are still some problems like image imperfection and noise interference, which seriously affect the performance of keyframe selection. Aiming at above problems, in this paper, we propose a linear reconstruction framework to summarize the videos. The first model in our framework seeks the most informative keyframes (base vectors) using the structure sparsity of the ℓ21 norm regularization, to represent all the frames as the linear combination of them in a video. Furthermore, we also propose another more robust model via ℓ21 norm based loss function to suppress the outlier, and form the joint sparsity with ℓ21 norm regularization. For the optimization, we design two efficient algorithms for two proposed models respectively. Finally the extensive experiments on real world video datesets are presented to show the effectiveness of the proposed framework.