Learning Low-Level Behaviors and High-Level Strategies in Humanoid Soccer

Abstract

This paper investigates the learning of both low-level behaviors for humanoid robot controllers and of high-level coordination strategies for teams of robots engaged in simulated soccer. Regarding controllers, current approaches typically hand-tune behaviors or optimize them without realistic constraints, for example allowing parts of the robot to intersect with others. This level of optimization often leads to low-performance behaviors. Regarding strategies, most are hand-tuned with arbitrary parameters (like agents moving to pre-defined positions on the field such that eventually they can score a goal) and the thorough analysis of learned strategies is often disregarded. This paper demonstrates how it is possible to use a distributed framework to learn both low-level behaviors, like sprinting and getting up, and high-level strategies, like a kick-off scenario, outperforming previous approaches in the FCPortugal3D Simulated Soccer team.