On Semantic Annotation for Sports Video Highlights by Mining User Comments from Live Broadcast Social Network

Abstract

In recent years, the idea of viewing online social media as human-powered sensing networks has draw significant attentions in research communities. Great examples are Twitter-based earthquake detection, Influenza detection, and traffic abnormally detection. Following the same viewpoint of the human-powered sensing network, in this paper, we discover the utility of user-generated social texts on social media platform for extracting highlights and annotating the semantics of sport video clips. The basic idea for the leverage of social text is that one can make use of the semantics of the social texts for understanding the corresponding moments of the game. For example, when watching a baseball, the users on social media will timely comments about the play, the team, and the events. By properly analyzing the texts, automatically annotating the sport videos turns out to be possible. However, two research challenges need to be addressed for such an idea: (1) as sport videos are often lengthy, how to precisely locate the moment of important events is a challenge task, (2) social media contents are generated by users on social network platform and contains various information and with noises, and therefore how to distill useful information from noisy social comment is also a challenge. In this paper, we present a weighting scheme to address the issues by estimating the importance of users (and therefore their comments) on social network platforms based on mining the interaction between users on social platforms. Also, we use soccer game videos and baseball game videos as well as social comment from on-line social network as our test data set. The evaluation over real data shows the effectiveness of the proposed framework.