An Approach to Decrease Interference among Modalities in Emotion Detection

Abstract

Emotion detection of video clips through multimodal information helps in a number of applications, including view mining, recommending system, public opinion and public sentiment monitoring. Current multimodal fusion techniques are applied to identify relevant information among modalities, which can introduce the interference of different modalities when detecting the emotion in the video. In this paper, we introduce a method which uses Q-Learning algorithm in attention-based network for decreasing interference among different emotion modalities. Experimental results prove that our method can improve the performance of basic attention-based network when detecting the emotion in the video.