Investigating LSTM for micro-expression recognition

Abstract

This study investigates the utility of Long Short-Term Memory (LSTM) networks for modelling spatial-temporal patterns for micro-expression recognition (MER). Micro-expressions are involuntary, short facial expressions, often of low intensity. RNNs have attracted a lot of attention in recent years for modelling temporal sequences. The RNN-LSTM combination to be highly effective results in many application areas. The proposed method combines the recent VGGFace2 model, basically a ResNet-50 CNN trained on the VGGFace2 dataset, with uni-directional and bi-directional LSTM to explore different ways modelling spatial-temporal facial patterns for MER. The Grad-CAM heat map visualisation is used in the training stages to determine the most appropriate layer of the VGGFace2 model for retraining. Experiments are conducted with pure VGGFace2, VGGFace2 + uni-directional LSTM, and VGGFace2 + Bi-directional LSTM on the SMIC database using 5-fold cross-validation.