Impression-aware video stream retrieval system with temporal color-sentiment analysis and visualization

Abstract

To retrieve Web video intuitively, the concept of "impression" is of great importance, because many users consider feelings and moods to be one of the most significant factors motivating them to watch videos. In this paper, we propose an impression-aware video stream retrieval system for querying the visual impression of video streams by analyzing the temporal change in sentiments. As a metric of visual impression, we construct a 180-dimensional vector space called as color-impression space; each dimension corresponds to a specific adjective representing humans' color perception. The main feature of this system is a context-dependent query processing mechanism to generate a ranking by considering the temporal transition of each video's visual impressions on viewers' emotion. We design an impression-aware noise reduction mechanism that dynamically reduces the number on non-zero features for each item mapped in the high-dimensional color-impression space by extracting the dominant salient impression features from a video stream. This system allows users to retrieve videos by submitting emotional queries such as "Find videos whose overall impression is happy and which have several sad and cool scenes". Through this query processing mechanism, users can effectively retrieve videos without requiring detailed information about them. © 2012 Springer-Verlag.