Real-time 3D hand gestures recognition for manipulation of industrial robots

Abstract

This paper presents an innovative system for industrial robot manipulation through 3D hand gestures. The interaction between the human operator and the robotic system is done via a dynamically created 3D virtual environment which is a precise copy of the robot's real working environment. The virtual environment enhances the manipulation of different objects and the selection of the desired action, in a natural way, only though hand gestures. Unlike most of the HRI based systems, ours is not meant for moving the robot or its mobile arm from one place to another, but to perform a specific task comprising of a list of specific operations. This kind of interaction has an important advantage: it eliminates the deadtimes which are specific to the direct-interaction based systems. The gesture recognition algorithm processes depth maps grabbed from a stereoscopic camera system and uses Dynamic Time Wrapping to compute the similarity between the hand trajectories acquired in real time and those from the gestures dictionary.