A natural interface to a virtual environment through computer vision-estimated pointing gestures

Abstract

This paper describes the development of a natural interface to a virtual environment. The interface is through a natural pointing gesture and replaces pointing devices which are normally used to interact with virtual environments. The pointing gesture is estimated in 3D using kinematic knowledge of the arm during pointing and monocular computer vision. The latter is used to extract the 2D position of the user’s hand and map it into 3D. Off-line tests show promising results with an average errors of 8cm when pointing at a screen 2m away.