Cross-Dimension Transfer Learning for Video-Based Facial Expression Recognition

Abstract

Dynamic Facial Expression Recognition (FER) in videos is currently a topic of broad concern. Considering the fact that 3-dimensional convolutional networks (3D ConvNets) have recently demonstrated poor performance in this task, we propose a simple, yet effective approach to solve this problem within limited emotion data, which we call cross-dimension transfer learning (CTL). By transferring parameters learned from 2D ConvNets into 3D, network can be initialized reasonably, making it possible to avoid training 3D ConvNets from scratch. We introduce several transfer strategies and experiment results show that, CTL methods can bring considerable improvement to 3D ConvNets and compared with training from scratch, recognition accuracy on AFEW (Acted Facial Emotion in the Wild) has improved by 12.79%. We further extend our method to CK+ (The Extended CohnKanade) dataset and the classification performance shows the generalized ability of our approach.