Control of a free-falling cat by policy-based reinforcement learning

Abstract

Autonomous control of nonholonomic systems is one big challenge, because there is no unified control method that can handle any nonholonomic systems even if the dynamics are known. To this challenge, in this study, we propose a reinforcement learning (RL) approach which enables the controller to acquire an appropriate control policy even without knowing the detailed dynamics. In particular, we focus on the control problem of a free-falling cat system whose dynamics are highly-nonlinear and nonholonomic. To accelerate the learning, we take the policy gradient method that exploits the basic knowledge of the system, and present an appropriate policy representation for the task. It is shown that this RL method achieves remarkably faster learning than that by the existing genetic algorithm-based method. © 2012 Springer-Verlag.