A combined strategy of hand tracking for desktop VR

Abstract

Desktop VR has been widely used in data analysis and VR movies. One of the important interactions in VR is to capture and track the 3D motion of hands. Although 3D hand pose estimation has been developed for many years, the trade-off between real-time and accuracy still exists. In this paper, we propose a strategy that combines fast model-based method and Convolutional Neural Network (CNN). Based on the occlusion of the hand depth image captured by Intel RealSense Camera, simple gesture images and complex gesture images are recognized by fast model-based method and CNN, respectively. A large number of experimental results demonstrate that our method achieves real-time performance with high accuracy.