An E-sports video highlight generator using win-loss probability model

Abstract

Highlighting, to compile key scenes in a match, is an essential task in the E-sports industry. Because E-sports is less dependent on time and place in its nature, there are countless matches every day. However, creating a highlight is intensive and time-consuming labor. Even the quality of the outcome depends on the editor's ability and decision. We propose a new approach to an automatic highlight generator from E-sports match videos. Our highlight generator reduces the cost of production and allows stable quality control. We defined the smallest component of the highlight as a 'Point', the moment when the win-loss probability of each team changes drastically. A 'Clip' is the contextual scene around Points, and a set of Clips is a 'Highlight'. Our highlight generator can prioritize Points, using a model that detects changes in real-time win-loss probabilities. It can create various versions of a highlight by adjusting the number of Points. Also, because it operates in real-time, we can generate highlights instantly. Our generator recognizes and extracts real-time state information from a match video using OpenCV and CNN. It uses an MLP model and gets the win-loss probability of each moment to calculate the change rate between directly adjacent moments. This MLP is pre-trained with match results in the past. A threshold is set by partially implementing CART algorithm for Gini Impurity. If a moment's change rate satisfies the threshold, it is classified as a Point. Then, a Clip is specified by setting the interval before and after the Point using the average interval. Finally, merging the Clips becomes a highlight. We created various versions of highlight for 119 match videos of E-Sports, the league of legend 2018 World Championship. To calculate the win-loss probability, the MLP pretrained the 11,082 match results. As a result, the accuracy and f1 scores were 89.9% and 74.5%, respectively, for the version most similar to the official highlight. We also compare our highlights by human evaluation. About 65% of reviewers said our highlights are better than official ones. In the exclusive Clip evaluations, the Clips exclusively included in ours earn 4.02 Point, but the Clips only in the official ones earn 2.63, which says our highlights include much better Clips than the official highlights.