Mechanical arm teleoperation control system by dynamic hand gesture recognition based on kinect device

Abstract

The research achieved to control the mechanical arm by using real‐time dynamic gesture recognition based on Kinect. It uses the unmarked gesture segmentation algorithm based on the palm neighbourhood and the threshold detection algorithm based on the palmar contour to identify the operator's gestures and moving trajectories, and to convert it into the specific action of the mechanical arm. Through wireless network, this system sends control instructions to the mechanical arm to realise teleoperation control. The system also provides video feedback of mechanical arm operation site. It improves the teleoperation telepresence and interactivity and avoids operation error such as grasp nothing or fall. The results of the experiment indicate that the operation of the gesture control system is simple and easy, the response of mechanical arm is quick and accurate and the human–machine interaction is intuitive and friendly.