Learning and Transfer of Movement Gaits Using Reinforcement Learning

Abstract

In this paper, a four-legged robot is trained to walk in the real world, without any manual engineering or programmed movement sequences. The goal is to enable robots to learn to walk on their own using reinforcement learning algorithms. Since each leg has three joints and the robot has four legs in total, this is a very complicated behavior to learn, so doing the training process in the real world would not be feasible. To accelerate the learning progress, an accurate simulated environment is used in which the robot can safely be trained using state-of-the-art learning algorithms, such as soft-actor critic and proximal policy optimization. The learnt behavior has then successfully been transferred to the real robot, with the real robot mirroring the behavior of the robot in the simulation. This resulted in the real robot moving forward. The Unity3D engine will be used for the simulation of the robot, along with the recently introduced ml-agents toolkit, which enable easy TensorFlow integration into the training environment. After a successful training in the simulation, the learned locomotion skills from the simulation is transferred to the robot in the real world. To transfer the data with minimal losses, the simulation environment will have to accurately mirror the real-world environment.