Video based emotion recognition using CNN and BRNN

Abstract

Video-based Emotion recognition is a rather challenging computer vision task. It not only needs to model spatial information of each image frame, but also requires considering temporal contextual correlations among sequential frames. For this purpose, we propose a hierarchical deep network architecture to extract high-level spatial-temporal features. In this architecture, two classic deep neural networks, convolutional neutral networks (CNN) and bi-directional recurrent neutral networks (BRNN), are employed to respectively capture facial textural characteristics in spatial domain and dynamic emotion changes in temporal domain. We endeavor to coordinate the two networks by optimizing each of them, so as to boost the performance of the emotion recognition. In the challenging competition, our method achieves a promising performance compared with the baselines.