Simulation for Control

Abstract

Computer simulation is a powerful tool for developing motion controllers in humanoid robotics and physics-based character animation. In this chapter, we first motivate simulation-based methods for the control of physical robots and digital avatars and discuss the major differences between developing controls in simulation and on real robots. We then detail a series of model-free algorithms developed by our group over the years. These algorithms are based on sampling time-indexed control actions for short-duration motion fragments, in order to track desired reference motion trajectories. We term the family of algorithms SAMCON (sampling-based motion control). The basic SAMCON and the improved SAMCON methods reconstruct open-loop controls; the reduced-order feedback policy learning and the guided SAMCON then build robust linear feedback strategies around the open-loop controls. We further introduce a general framework – control graphs – that learns and organizes multiple physics-based motion skills and their transitions from a set of example motion capture clips. This offers a potential solution for the development of rich motion repertoires for humanoid robots as well as encourages widespread adoption of physics-based methods for character animation. Finally, we discuss research opportunities for future research on simulation-based control methods and transferring such controls to physical robots.