Intelligent multi-fingered dexterous hand using virtual reality (VR) and robot operating system (ROS)

Abstract

The concept of using robots for reducing human effort is now a buzzword since the last decade. Now, with the advancement of the current technologies the robots could be remotely controlled with much greater ease and precision than before in domains, where we could have never even thought of their applications. The paper describes a new user friendly interface which allows the control of the robot manipulator (hand) in the most effective and simplest manner possible. By making use of 360° camera the vision is created which is displayed on the user’s mobile phone (which is placed inside a VR-goggles) and viewed as a virtual reality environment by the user. The user and the robot manipulator are situated far apart and the user will feel like as if the robotic arms are his own arms. Moreover, the user can control the robotic arm placed remotely in any high risk area with or without any internet connectivity. This methodology is very useful for controlling the robotic arm for dealing with radioactive and hazardous materials. Therefore, the user doesn’t have to be in the place of risk and he/she is still able to operate in the most convenient manner. In this paper, a detailed description of the control of the open-source 3D printed robotic hand/forearm from InMoov is discussed. Basically, the InMoov hand/forearm is modified into a useable all-purpose dexterous hand with the application of Virtual Reality to enhance user ease. There are two modes in which the system could be used. First is the hand mimicking mode, in which the user’s hand movements are being replicated onto the robotic hand/forearm, and the second mode is the voice recognition mode in which simple voice commands from the user are processed and are used to control the robotic arm. Voice recognition enables hands free usage of the robotic hand for people with disability. The system makes use of ROS (Robot Operating System) as its baseline program management system. The development is done by making use of ROS catkin workspace on Raspberry Pi 3 with Ubuntu Mate with the implementation of the necessary packages and nodes for every functionality.