Query-Based Video Summarization System Based on Light Weight Deep Learning Model

Abstract

Video summarization technologies strive to provide a succinct and thorough description by selecting the most informative frames from the original video. The essential concept of query-based video summarization methods is represented by constructing a video summary related to user query in term of user interest. This paper aims to address the problem of query-based video summarization construction task by adopting a lightweight deep learning model for object detection task in order to evident how concerning a frame or shot is to a given query. Both YOLOv3 and Tiny YOLOv3 deep learning models are used separately in the proposed system to train the networks using images and videos dataset including diverse types of objects such as; (motorcycles, bicycles, cars, buses, and trucks). Subsequence, the most relative frames to a given query are selected and assembled as keyframes using a modified K-mean clustering scheme to provide different interesting summaries from the original video. Based on the experimental results of object detection phase, we have obtained an average object detection accuracy around 93%, 83% based on YOLOv3 and Tiny YOLOv3 deep learning models respectively. Comprehensive experiments were performed to evaluate the proposed video summarization system, which exhibited an efficient summarization rate close to 33% of the original video. Further experiments were conducted using standard UTE dataset to exhibit the competitive performance of the proposed method compared to state of art query-based video summarization methods