Recognition of facial action units with action unit classifiers and an association network

Abstract

Most previous work of facial action recognition focused only on verifying whether a certain facial action unit appeared or not on a face image. In this paper, we report our investigation on the semantic relationships of facial action units and introduce a novel method for facial action unit recognition based on action unit classifiers and a Bayes network called Facial Action Unit Association Network (FAUAN). Compared with other methods, the proposed method attempts to identify a set of facial action units of a face image simultaneously. We achieve this goal by three steps. At first, the histogram of oriented gradients (HOG) is extracted as features and after that, a Multi-Layer Perceptron (MLP) is trained for the preliminary detection of each individual facial action unit. At last, FAUAN fuses the responses of all the facial action unit classifiers to determine a best set of facial action units. The proposed method achieves a promising performance on the extended Cohn-Kanade Dataset. Experimental results also show that when the individual unit classifiers are not so good, the performance could improve by nearly 10% in some cases when FAUAN is used.