Path Navigation for Multi-Agent Flight System Based on Distributed Model Predictive Control and Reinforcement Learning

Abstract

The continuous change in formation configurations offers significant potential for formation flight. However, existing methods neglect the potential of autonomous navigation and learning ability based on environmental conditions, which would limit scalability for large formations to some extent. To address this, this paper presents an integrated framework that incorporates an innovative policy for configuration learning into multi-agent flight systems. Firstly, a decoupled trajectory optimization model is proposed, consisting of prediction and learning modules that ensure local adaptivity, obstacle avoidance and dynamic feasibility of formation configuration. Secondly, we design a stage cost function representing obstacle conditions, configuration variability and minimum distance constraints between agents. Then, to enhance adaptability in constrained scenarios, we introduce a reference value generation method that adjusts trajectory learning based on local observations, penalizing errors. Finally, we propose a configuration learning strategy called the local-level action planner and global cost, which coordinates local trajectory optimization with global maintenance. This method integrates distributed model predictive control (DMPC) with modified multi-agent proximal policy optimization (MAPPO), easing the challenge of tracking trajectories from varying rigid geometric shapes. Simulation results demonstrate that the proposed approach improves success rate compared to traditional methods while maintaining formation and avoiding obstacles.