Enhanced Gaze Following via Object Detection and Human Pose Estimation

Abstract

The aim of gaze following is to estimate the gaze direction, which is useful for the understanding of human behaviour in various applications. However, it is still an open problem that has not been fully studied. In this paper, we present a novel framework for gaze following problem, where both the front/side face case and the back face case are taken into account. For the front/side face case, head pose estimation is applied to estimate the gaze, and then object detection is used to further refine the gaze direction by selecting the object that intersects with the gaze in a certain range. For the back face case, a deep neural network with the human pose information is proposed for gaze estimation. Experiments are carried out to demonstrate the superiority of the proposed method, as compared with the state-of-the-art method.