Training a RoboCup Striker Agent via Transferred Reinforcement Learning

Abstract

Recent developments in reinforcement learning algorithms have made it possible to train agents in highly complex state and action spaces, including action spaces with continuous parameters. Advancements such as the Deep-Q Network and the Deep Deterministic Policy Gradient were a critical step in making reinforcement learning a feasible option for training agents in real world scenarios. The viability of these technologies has previously been demonstrated in training a RoboCup Soccer agent with no prior domain knowledge to successfully score goals; however, this work required an engineered intermediate reward system to direct the agent in its exploration of the environment. We introduce the use of transfer learning rather than engineered rewards. Our results are positive, showing that it is possible to train an agent through a series of increasingly difficult tasks with fewer training iterations than with an engineered reward. However, when the agent’s likelihood of success in a task is low, it may be necessary to reintroduce an engineered reward or to provide extended training and exploration using simpler tasks.