Towards Self-Adaptive Resilient Swarms Using Multi-Agent Reinforcement Learning

Abstract

Cooperative swarms of intelligent agents have been used recently in several different fields of application. The ability to have several units working together to accomplish a task can drastically extend the range of challenges that can be solved. However, these swarms are composed of machines that are susceptible to suffering external attacks or even internal failures. In cases where some of the elements of the swarm fail, the others must be capable of adjusting to the malfunctions of the teammates and still achieve the objectives. In this paper, we investigate the impact of possible malfunctions in swarms of cooperative agents through the use of Multi-Agent Reinforcement Learning (MARL). More specifically, we investigate how MARL agents react when one or more teammates start acting abnormally during their training and how that transfers to testing. Our results show that, while common MARL methods might be able to adjust to simple flaws, they do not adapt well when these become more complex. In this sense, we show how independent learners can be used as a potential direction of future research to adapt to malfunctions in swarms using MARL. With this work, we hope to motivate further research to create more robust intelligent swarms using MARL.