Eye-state action unit detection by Gabor wavelets

Abstract

Eyes play important roles in emotion and paralinguistic communications. Detection of eye state is necessary for applications such as driver awareness systems. In this paper, we develop an automatic system to detect eye-state action units (AU) based on Facial Action Coding System (FACS) by use of Gabor wavelets in a nearly frontal-viewed image sequence. Three eye-state AU (AU 41, AU42, and AU43) are detected. After tracking the eye corners in the whole sequence, the eye appearance information is extracted at three points of each eye (i.e., inner corner, outer corner, and the point between the inner corner and the outer corner) as a set of multi-scale and multi-orientation Gabor coefficients. Then, the normalized Gabor coefficients are fed into a neural-network-based eye state AU detector. An average recognition rate of 83%is obtained for 112 images from 17 image sequences of 12 subjects.