Preserving Privacy of Face and Facial Expression in Computer Vision Data Collected in Learning Environments

Abstract

Learners’ affective states are analyzed using their facial expressions obtained from image/video data. Deep learning architectures are the state-of-the-art affective state recognition methods used in the literature. Vast volumes of these data are used in the training, testing, and validation process. Now, sharing these data publicly for research purposes is a challenging task as it has privacy issues. The importance of creating a successful face de-identification algorithm for privacy protection cannot be overstated because the face is the single biometric feature that discloses the most recognizable qualities of a person in an image or a video frame. Existing approaches to face de-identification with facial expressions are not explored in the education domain. In this study, we design a methodology for a face de-identification technique that automatically creates a new face while maintaining the emotion and non-biometric facial characteristics of a target face from an input facial image. We consider a proxy set, which consists of a sizable number of synthetic faces produced by StyleGAN, and choose the proxy set face that most closely resembles the target face in terms of facial expression and position. Since the faces in the proxy settings were formed artificially, the face chosen from this collection is absolutely anonymous. We created a dataset of 10 students to test the methodology. The performance of StyleGAN was measured for standard parameters such as gender, emotion, age, etc., and the results show that the generated face preserved emotional attributes with a de-identified face.