Gesture-Controlled Robotic Arm for Small Assembly Lines

Abstract

In this study, we present a gesture-controlled robotic arm system for small assembly lines. Robotic arms are extensively used in industrial applications; however, they typically require special treatment and qualified personnel to set up and operate them. Towards this end, hand gestures can provide a natural way for human–robot interaction, providing a straightforward means for control without the need for significant training of the operators. Our goal is to develop a safe, low-cost, and user-friendly system for environments that often involve non-repetitive and custom automation processes, such as in small factory setups. Our system estimates the 3D position of the user’s joints in real time with the help of AI and real-world data provided by an RGB-D camera. Then, joint coordinates are translated into the robotic arm’s desired poses in a simulated environment (ROS), thus achieving gesture control. Through the experiments we conducted, we show that the system provides the performance required to control a robotic arm effectively and efficiently.