Acquisition of Cooperative Behavior in a Soccer Task Using Reward Shaping

Abstract

In this research, soccer task is investigated among the numerous tasks of deep reinforcement learning. The soccer task requires cooperative behavior. However, it is difficult for the agents to acquire the behavior, because a reward is sparsely given. Moreover, the behaviors of the allies and opponents must be considered by the agents. In addition, in the soccer task, if the agents attempt to acquire high-level cooperative behavior from low-level movements, such as ball kicking, a huge amount of time will be needed to learn a model. In this research, we conduct experiments in which reward shaping is incorporated into deep reinforcement learning. This enables the agents to efficiently acquire cooperative behavior from low-level movements in a soccer task. The findings of this research indicate that reward shaping with a designer's domain knowledge positively influences the agent's attempt to acquire cooperative behavior from low-level movements.