Eye contact with a virtual character using a vision-based head tracker

Abstract

Eye contact with virtual character can provide a realistic illusion to a user in an immersive virtual reality (VR) environment. This allows more believable eye communication between a user and a computer-generated virtual character. In this paper, an effective eye contact mechanism by innovatively combining a vision-based head tracker and an eye animation method is shown. Together, a robust color classification method to track the user's face is developed which includes extracting the 3D information of the tracked face available from a stereo camera without the need for any handheld devices or special sensors attached on the user. Through the use of the vision-based head tracker, the virtual character will be aware of the user's movements and can react to the user in a believable and pleasing manner by moving the head and the eyes to gaze at the user when the user is looking at the virtual character, or turning away when the user is not looking, or be in idle mode when the user is not around. The proposed eye contact method has been successfully applied in a virtual reality interactive game. © 2011 Springer Science+Business Media B.V.