Variable Action Set-Based Monte Carlo Tree Search Algorithm for AAV Autonomous Collision Avoidance

Abstract

Autonomous collision avoidance of Autonomous Aerial Vehicles (AAVs) is becoming a hot research topic in Urban Air Mobility (UAM) operations. Many studies modeled this problem as a Markov decision process (MDP) and adopted intelligent methods such as reinforcement learning and Monte Carlo Tree Search (MCTS) to design collision avoidance algorithms. Among these studies, some algorithms with discrete action sets have achieved good results in collision avoidance. However, the fixed discrete action set often leads to low flight quality, as the rigid control makes the AAV unable to stably fly straight in the desired direction but can only fly wobbly. To address this problem, a Variable Action Set-based MCTS algorithm (VASM) for autonomous collision avoidance is proposed in this paper. By introducing an additional action option that is carefully designed to improve the stability and efficiency of flight along any desired heading, a variable action set is constructed for VASM. In addition, a reward function that takes into account more rational evaluation factors is also proposed to facilitate better decision-making. We conducted numerical experiments to verify the effectiveness of VASM, and results show that compared with the conventional MCTS algorithm, VASM can better improve flight control and flight quality with only a small increase in computational cost, achieving more efficient and energy-saving flight guidance. Specifically, in high-density airspace, VASM reduces average flight distance by 15.34%, average flight time by 15.08%, average flight turns by 36.64%, conflict probability by 18.56%, while increasing target arrival probability to 98.3%.