Augmenting sampling based controllers with machine learning

Abstract

Efficient learning of 3D character control still remains an open problem despite of the remarkable recent advances in the field. We propose a new algorithm that combines planning by a sampling-based model-predictive controller and learning from the planned control, which is very noisy. We combine two methods of learning: 1) immediate but imprecise nearest-neighbor learning, and 2) slower but more precise neural network learning. The nearest neighbor learning allows to rapidly latch on to new experiences whilst the neural network learns more gradually and develops a stable representation of the data. Our experiments indicate that the learners augment each other, and allow rapid discovery and refinement of complex skills such as 3D bipedal locomotion. We demonstrate this in locomotion of 1-, 2- and 4-legged 3D characters under disturbances such as heavy projectile hits and abruptly changing target direction. When augmented with the learners, the sampling based model predictive controller can produce these stable gaits in under a minute on a 4-core CPU. During training the system runs real-time or at interactive frame rates depending on the character complexity.