Transfer Learning in Facial Emotion Recognition: Useful or Misleading?

Abstract

The use of machine learning in medicine holds a lot of potential in the domains of patient diagnosis, monitoring and treatment. One such application is the use of Emotion intelligence to aid in the treatment of people suffering from autism spectrum disorder (ASD). As training a robust network model requires large datasets, transfer learning is often implemented. The aim of this study is to show if using pretrained weights, trained on different images, as an initial starting point for training a new model remains biased after training and does not generalize well to unseen data of the new trained model. Image pre-processing was performed and the data trained on three models, the base model of VGG16 architecture and two with attention modules of SE and CBAM. The OULU-CASIA database was used for training with 10-fold cross validation for evaluating the performance of the model and the robustness was tested against two other emotion datasets of FACES and JAFFE. The results showed the training from scratch had better adherence to the regions of importance in the image. This validates the hypothesis that prior knowledge, i.e. weights from pre-trained models of large datasets, may not be usable for special applications.