Compact, Accurate and Low-cost Hand Tracking System based on LEAP Motion Controllers and Raspberry Pi

Abstract

The large diffusion of low-cost computer vision (CV) hand tracking sensors used for hand gesture recognition,has allowed the development of precise and low cost touchless tracking systems. The main problem with CVsolutions is how to cope with occlusions, very frequent when the hand has to grasp a tool, and self-occlusionsoccurring when some joint obscures some other. In most cases occlusions are solved by using synchronizedmultiple stereo sensors. Virtual Glove (VG) is one of the CV-based systems that uses two orthogonal LEAPsensors integrated into a single system. The VG system is driven by a Personal Computer in which both amaster operating system (OS) and a virtual machine have to be installed in order to drive the two sensors(just one sensor at a time can be driven by a single OS instance). This is a strong limitation because VG hasto run on a powerful PC, thus resulting in a not properly low-cost and portable solution. We propose a VGarchitecture based on three Raspberry Pi (RP), each consisting of a cheap single board computer with LinuxOS. The proposed architecture assigns an RPi to each LEAP and a third RP to collect data from the other two.The third RP merges, in real time, data into a single hand model and makes it available, through an API, tobe rendered in a web application or inside a Virtual Reality (VR) interface. The detailed design is proposed,the architecture is implemented and experimental benchmark measurements, demonstrating the RPi-basedVG real-time behaviour while containing costs and power consumption, are presented and discussed. Theproposed architecture could open the way to develop modular hand tracking systems based on more than twoLEAPs, each associated to one RP, in order to further improve robustness