Communication-Aware Graph Neural Network for Multi-Agent Reinforcement Learning

Abstract

Multi-agent reinforcement learning (MARL) requires effective communication strategies to solve complex control tasks over uncertain communication channels. This paper explores a communication-aware graph neural network (GNN) approach for MARL, where the interactions between agents are modeled as a dynamic directed graph that explicitly considers time-varying lossy links. We integrate communication aspects into MARL by combining the self-attention-based coordination graph and a graph convolution with zero-input compensation to migrate the information losses over multi-hop networks. We evaluate our approach on two challenging tasks: the predator-prey and the coverage problems. We show 1) the operational benefits of communication-aware GNN with sensing range, node density, and task complexity, 2) the robust performance of the proposed scheme to support the graph convolution over various ranges of packet loss probabilities of links, and 3) the effectiveness of the residual connection of the GNN model on the overall performance and the communication architecture.