Building the facial expressions recognition system based on RGB-D images in high performance

Abstract

In this paper, we propose a novel idea for automatic facial expression analysis with the aim of resolving the existing challenges in 2D images. The subtle combination of the geometry-based method with the appearance-based features in depth and color images contributes to increasing in distinguishable features among various facial expressions. Particular functions are utilised to calculate the correlation between expressions in order to determine the exact facial expression. Our approach consists of a sequence of steps including estimating the normal vector of facial surface, then extracting the geometric features such as the orientation of normal vector in the point cloud. The useful color information is known as LBP. According to the result of the experiment, we demonstrate that the effective fusion scheme of texture and shape feature on color and depth images. In comparison with the non fusion scheme, our fusion scheme has resulted in the increase of recognition under low and high illuminated light, about 19.84% and 1.59 %, respectively.