Model-Based Actor-Critic Learning of Robotic Impedance Control in Complex Interactive Environment

Abstract

In complex robot applications, such as human-robot interaction and robot machining, robots should interact with an unknown environment. To learn the interactive skill, a model-based actor-critic learning algorithm and a safety-learning strategy are proposed in this article to find the optimal impedance control, in which the learning process is safe and fully automatic and does not know the system parameter. In the learning algorithm, a critic is defined as a quadratic form of the system states and the external force. A modified deterministic policy gradient algorithm is presented to improve the learning efficiency. The proposed approach utilizes a model-based constraint and a highly efficient learning algorithm. In the safety-learning strategy, the robot is trained under a constant force, and the learned impedance control can transfer to different interaction situations by choosing the suitable impedance index. The effectiveness of the learning algorithm and the performance of the learned impedance control are validated in a UR5 robot. The robot can perform human-robot interaction and robot machining tasks after the training process with 100 s training time.