3D Pose Estimation of Robot Arm with RGB Images Based on Deep Learning

Abstract

In the field of human-robot interaction, robot collision avoidance with the human in a shared workspace remains a challenge. Many researchers use visual methods to detect the collision between robots and obstacles on the assumption that the robot pose is known because the information about the robot is obtained from the controller and hand-eye calibration is conducted. Therefore, they focus on the motion prediction of obstacles. In this paper, a real-time method based on deep learning is proposed to directly estimate the 3D pose of the robot arm using a color image. The method aims to remove the hand-eye calibration when the system needs to be reconfigured and increase the flexibility of the system by eliminating the requirement that the camera fixed relative to the robot. Our approach has two main contributions. One is that the method estimates the 3D position of the robot base and the relative 3D positions of the predefined key points of the robot to the robot base separately different from other deep learning methods considering the limitations of the dataset. The other is that some datasets are collected through another trained network to avoid tedious calibration process, and the trained network will be reused in the pose estimation task. Finally, the experiments are conducted. The results show that a fully trained system provides an accurate 3D pose estimation for the robot arm in the camera coordinate system. The average errors of the 3D positions of the robot base and the predefined key points are 2.35 cm and 1.99 cm respectively.