Exploiting multimodal affect and semantics to identify politically persuasive web videos

Abstract

We introduce the task of automatically classifying politically persuasive web videos and propose a highly effective multimodal approach for this task. We extract audio, visual, and textual features that attempt to capture affect and semantics in the audio-visual content and sentiment in the viewers' comments. We demonstrate that each of the feature modalities can be used to classify politically persuasive content, and that fusing them leads to the best performance. We also perform experiments to examine human accuracy and inter-coder reliability for this task and show that our best automatic classifier slightly outperforms average human performance. Finally we show that politically persuasive videos generate more strongly negative viewer comments than nonpersuasive videos and analyze how affective content can be used to predict viewer reactions.