Deep Learning Based Automated Sports Video Summarization using YOLO

Abstract

This paper proposes a computationally inexpensive method for automatic key-event extraction and subsequent summarization of sports videos using scoreboard detection. A database consisting of 1300 images was used to train (using transfer learning) a supervised-learning based object detection algorithm, YOLO (You Only Look Once). Then, for each frame of the video, once the scoreboard was detected using YOLO, the scoreboard was cropped out of the image. After this, image processing techniques were applied on the cropped scoreboard to reduce noise and false positives. Finally, the processed image was passed through an OCR (Optical Character Recognizer) to get the score. A rule-based algorithm was run on the output of the OCR to generate the timestamps of key-events based on the game. The proposed method is best suited for people who want to analyse the games and want precise timestamps of the occurrence of important events. The performance of the proposed design was tested on videos of Bundesliga, English Premier League, ICC WC 2019, IPL 2019, and Pro Kabaddi League. An average F1 Score of 0.979 was achieved during the simulations. The algorithm is trained on five different classes of three separate games (Soccer, Cricket, Kabaddi). The design is implemented using python 3.7.