Levels of realism for cooperative multi-agent reinforcement learning

Abstract

Training agents in a virtual crowd to achieve a task can be accomplished by allowing the agents to learn by trial-and-error and by sharing information with other agents. Since sharing enables agents to potentially reach optimal behavior more quickly, what type of sharing is best to use to achieve the quickest learning times? This paper categorizes sharing into three categories: realistic, unrealistic, and no sharing. Realistic sharing is defined as sharing that takes place amongst agents within close proximity and unrealistic sharing allows agents to share regardless of physical location. This paper demonstrates that all sharing methods converge to similar policies and that the differences between the methods are determined by analyzing the learning rates, communication frequencies, and total run times. Results show that the unrealistic-centralized sharing method - where agents update a common learning module - is the most effective of the sharing methods tested. © 2012 Springer-Verlag.