Towards Intelligent Unmanned Adversarial Games: A Reinforcement Learning Framework with the PHP-ROW Method

Abstract

This study introduces a novel framework for intelligent unmanned BVR maneuver control within the context of adversarial games. The emphasis lies on three pivotal aspects: situational awareness, maneuver decision-making, and precise maneuver control. Within this paradigm, our unmanned aerial vehicles (UAVs) can assimilate crucial situational information through constructed situational vectors and execute sophisticated maneuvers, effectively addressing the intricacies of dynamic flight environments and various unpredictable scenarios within the game setting. To achieve granular maneuver control, this research introduces the Priority Heading Polling–Random Observation Weight (PHP-ROW) method, underpinned by deep reinforcement learning. This approach integrates two primary components: (1) the priority heading polling (PHP) mechanism, which governs the extent of flight trajectories while emphasizing heading control, and (2) the random observation weight (ROW) technique, which adeptly moderates the influence of roll angle rewards during the learning phase. The superiority of the PHP-ROW method is showcased by contrasting it against the conventional proximal policy optimization (PPO) algorithm. Conclusively, the utility and efficacy of the presented framework are corroborated through human–machine adversarial game simulations in a hyper-realistic environment. This investigation provides foundational theoretical and empirical contributions to the realm of intelligent unmanned aerial maneuver control, promising significant implications for the evolution of aviation technology in adversarial game contexts.