Image Augmentation for Eye Contact Detection Based on Combination of Pre-trained Alex-Net CNN and SVM

Abstract

Making eye contact is the most powerful mode of establishing a communicative link between humans. We propose a method for detecting eye contact (mutual gaze) from images of both eyes through the combined usage of a pre-trained convolutional neural network (CNN) and a support vector machine (SVM). Neural networks are a powerful technology for classifying object images. When it comes to classification accuracy, a huge number of training samples is the key to success. The training samples are augmented by image perturbation, namely, shifting the cropping regions. A pre-trained CNN, Alex-Net, is used as the image feature extractor after being pre-trained for large-scale object image datasets. An SVM is used as the trainable classifier. Original both-eyes samples of two classes on the Columbia Gaze Data Set CAVE-DB are divided in five-fold cross-validation. Manually cropped images and automatically augmented images on the CAVE-DB are trained by the SVM. The feature vectors of the eye images are then passed to the SVM from Alex-Net. We performed 5-fold t-testing on 77 images and found that the average error rate was 16.44%, and the lowest error rate of images without glasses was 8.96% with 7,850 training images of perturbation. These results demonstrate that the proposed method is effective in detecting eye contact.