WRFMR: A Multi-Agent Reinforcement Learning Method for Cooperative Tasks

Abstract

Multi-agent reinforcement learning (MARL) for cooperative tasks has been extensively studied in recent years. The balance of exploration and exploitation is crucial to MARL algorithms' performance in terms of the learning speed and the quality of the obtained strategy. To this end, we propose an algorithm known as the weighted relative frequency of obtaining the maximal reward (WRFMR), which uses a weight parameter and the action probability to balance exploration and exploitation and accelerate convergence to the optimal joint action. For the WRFMR algorithm, each agent needs to share the state and the immediate reward and does not need to observe the actions of the other agents. Theoretical analysis on the model of WRFMR in cooperative repeated games shows that each optimal joint action is an asymptotically stable critical point if the component action of every optimal joint action is unique. The box-pushing task, the distributed sensor network (DSN) task, and a strategy game known as blood battlefield are used for empirical studies. Both the DSN task and the box-pushing task involve full cooperation, while blood battle comprises both cooperation and competition. The simulation results show that the WRFMR algorithm outperforms the other algorithms regarding the success rate and the learning speed.