Characterizing the Performance of Deep Neural Networks for Eye-Tracking

Abstract

Deep neural networks (DNNs) provide powerful tools to identify and track features of interest, and have recently come into use for eye-Tracking. Here, we test the ability of a DNN to predict keypoints localizing the eyelid and pupil under the types of challenging image variability that occur in mobile eye-Tracking. We simulate varying degrees of perturbation for five common sources of image variation in mobile eye-Tracking: rotations, blur, exposure, reflection, and compression artifacts. To compare the relative performance decrease across domains in a common space of image variation, we used features derived from a DNN (ResNet50) to compute the distance of each perturbed video from the videos used to train our DNN. We found that increasing cosine distance from the training distribution was associated with monotonic decreases in model performance in all domains. These results suggest ways to optimize the selection of diverse images for model training.