Underwater Multiple AUV Cooperative Target Tracking Based on Minimal Reward Participation-Embedded MARL

Abstract

Recently, the rapid advancement of Multi-Agent Reinforcement Learning (MARL) has introduced a new paradigm for intelligent underwater target tracking within Autonomous Underwater Vehicle (AUV) cluster networks, enabling these networks to intelligently collaborate in target tracking. However, the limited scalability of MARL poses significant challenges to the performance of AUV cluster networks in tracking tasks. Specifically, MARL models trained on a fixed agents lose their effectiveness when the agent count changes, underscoring the critical need to enhance MARL's scalability to accommodate an arbitrary number of agents. This paper addresses the pressing issue of MARL's scalability in the context of AUV cluster network-based target tracking. Specifically, we propose an Elastic Software-Defined Multi-Agent Reinforcement Learning (ESD-MARL) architecture to enhance the scalability of AUV cluster networks. Moreover, we propose an Incremental Multi-Agent Reinforcement Learning algorithm based on Minimal Reward Participation (IMARL-MRP) that allows for the expansion of the agents without retraining. By integrating the ESD-MARL with the IMARL-MRP, we propose an elastic underwater target tracking scheme, achieving high-performance target tracking with enhanced scalability. Evaluation results demonstrate that the proposed approach effectively enhances the scalability of MARL, enabling the arbitrary expansion of the AUV cluster network, thus supporting scalable and efficient underwater target tracking.