Facial expression recognition adaptive to face pose using RGB-D camera

Abstract

In this paper, we propose a facial expression recognition method for non-frontal faces using RGB-D camera. The method uses the depth information of the RGB-D camera to calculate the face pose, modeled using a cylinder. Feature points obtained by the RGB-D camera, modified by the face pose, are compared with Action Units of the Facial Action Coding System for recognition of facial expression. Experiments were conducted using facial images in three types of angles and four expressions: anger, sadness, happiness, and surprise. Results of the experiments have shown that the method is rather robust to roll rotations than yaw and pitch rotations.