Deep Reinforcement Learning with Application to Air Confrontation Intelligent Decision-Making of Manned/Unmanned Aerial Vehicle Cooperative System

Abstract

With the development of intelligence in air confrontation, the demand for cooperative engagement of manned/unmanned aerial vehicle (MAV/UAV) is becoming more intense. Deep reinforcement learning (DRL), which combines the abstract representation capability of deep learning (DL) and the optimal decision-making and control capability of reinforcement learning (RL), is an appropriate application for dealing with this problem. In the case of continuous action space, the dynamics model of UAV and the basic structure of one of the most popular DRL methods called deep deterministic policy gradient (DDPG) are built firstly. To establish the framework of intelligent decision-making of MAV/UAV, typical intentions including Head-on attack, Fleeing, Pursuing and Energy-storing, corresponding to four optimization models, are introduced secondly. Then the neural network is trained by means of reconstructing the replay buffer of DDPG algorithm. Finally, simulation results show that UAV is able to learn intelligent decision-making throughout the intention guiding of MAV. Compared with original DDPG algorithm, the improved method can achieve a better performance in convergence and stability. Furthermore, the level of intelligent decision-making in air confrontation can be improved by self-learning.