Enhancement of Control Performance for Degraded Robot Manipulators Using Digital Twin and Proximal Policy Optimization

Abstract

In this study, we propose a novel method for enhancing the control performance of a degraded robot manipulator by leveraging digital twins and proximal policy optimization, a specific deep reinforcement learning algorithm. Recently, various robotic technologies with high levels of controllability, safety, and reliability that incorporate the fourth industrial technology have been developed. Nevertheless, repairs or replacements owing to the performance degradation of sophisticated robot hardware or control systems are still time-, cost-, and manpower-consuming. To address these challenges, we propose a new strategy:1) approximate the unstable dynamic characteristics of six-degree-of-freedom low-performance robot manipulators to a digital twin with parameter tuning of a physics engine; 2) improve the accuracy and stability of reaching target points under diverse conditions through deep reinforcement learning using the domain randomization method; and 3) deploy a trained policy on an actual robot manipulator with degraded capabilities to validate the control performance improvement. Our method reduced the position error of a real robot manipulator by 63.0% and 39.0% compared to the built-in control method and the proportional-integral-derivative control method, respectively. Randomizing parameters in the physics engine of the digital twin during training allowed the method to simulate the imprecise motions of an actual degraded robot manipulator, facilitating the development of a more robust policy. Notably, our method has the potential to be applicable to all types of articulated robots, and presents a promising solution for maintaining performance while reducing long-term costs.