Robust depth camera based eye localization for human-machine interactions

Abstract

This paper presents a novel approach to depth camera based single-/multi-person eye localization for human-machine interactions. Intensity and depth image frames of a single depth camera are used as system input. Foreground objects are segmented respectively from the depth image by using a novel object segmentation technique based on 2-D histogram with Otsu's method. Contour analysis with ellipse fitting is performed to locate the potential face region on the detected object. Finally, an eye localization algorithm based on a predefined eye template and geometric features is applied on the extracted facial sub-images, which is a hybrid solution combining appearance and feature based eye detection methods using SVM classification to gain robustness. Our goal is to realize a low-cost and robust machine vision system which is insensitive to low spatial resolution for eye detection and tracking based applications, e.g., driver drowsiness detection, autostereoscopic display for gaming/home/office use. The experimental results of the current work with ARTTS 3-D TOF database and with our own Kinect image database demonstrate that the average eye localization rate per face is more than 92% despite of illumination change, head pose, facial expression and spectacles. The performance can be further improved with the integration of an effective tracking algorithm. © 2011 Springer-Verlag.