Eye gaze correction for video conferencing using Kinect v2

Abstract

In video conferencing, eye gaze correction is beneficial for effective communication. In this era, video conferencing at homes using laptops is straightforward. In this paper, we propose an eye gaze correction method with a low-cost simple setup using Kinect v2. Our method detects an ellipse that connects edge points of the face after identifying several feature points within the face using Kinect v2 SDK. Then, we apply a 3D affine transform that allows eye gaze correction using camera space points that are acquired from depth information. Thus, in the preprocessing step, an ellipse model should be extracted when the user gazes the camera and display, respectively. Also, we fill holes that are caused by the affine transform using color inpainting. As a result, we produced a natural eye gaze-corrected image in real-time.