Heterogeneous Knowledge Transfer in Video Emotion Recognition, Attribution and Summarization Transactions on Affective Computing Heterogeneous Knowledge Transfer in Video Emotion Recognition, Attribution and Summarization

  • Xu B
  • Fu Y
  • Jiang Y
 et al. 
  • 1

    Readers

    Mendeley users who have this article in their library.
  • N/A

    Citations

    Citations of this article.

Abstract

—Emotional content is a key element in user-generated videos. However, it is difficult to understand emotions conveyed in such videos due to the complex and unstructured nature of user-generated content and the sparsity of video frames that express emotion. In this paper, for the first time, we study the problem of transferring knowledge from heterogeneous external sources, including image and textual data, to facilitate three related tasks in video emotion understanding: emotion recognition, emotion attribution and emotion-oriented summarization. Specifically, our framework (1) learns a video encoding from an auxiliary emotional image dataset in order to improve supervised video emotion recognition, and (2) transfers knowledge from an auxiliary textual corpus for zero-shot recognition of emotion classes unseen during training. The proposed technique for knowledge transfer facilitates novel applications of emotion attribution and emotion-oriented summarization. A comprehensive set of experiments on multiple datasets demonstrate the effectiveness of our framework.

Get free article suggestions today

Mendeley saves you time finding and organizing research

Sign up here
Already have an account ?Sign in

Find this document

Authors

  • Baohan Xu

  • Yanwei Fu

  • Yu-Gang Jiang

  • Boyang Li

  • Leonid Sigal

Cite this document

Choose a citation style from the tabs below

Save time finding and organizing research with Mendeley

Sign up for free