Multi-agent cooperative reinforcement learning in 3D virtual world

Abstract

A virtual world is an online community in the form of a computer-based simulated environment, through which users can interact with one another and use and create objects. The non-player characters (NPC) in virtual world are following a fixed set of pre-programmed behaviors and lack the ability to adapt with the changing surrounding. Reinforcement learning agent is a way to deal with this problem. However, in a cooperative social environment, NPC should learn not only by trial and error, but also through cooperation by sharing information. The key investigation of this paper is: modeling the NPCs as multi-agent, and enable them to conduct cooperative learning, then speeding up the learning process. By using a fire fighting scenario in Robocup Rescue, our research shows that sharing information between cooperative agents will outperform independent agents who do not communicate during learning. The further work and some important issues of multi-agent reinforcement learning in virtual world will also be discussed in this paper. © 2010 Springer-Verlag.