Supplementary Reinforcement Learning Controller Designed for Quadrotor UAVs

Abstract

The control problem for quadrotor UAVs is difficult and challenging due to the complex nonlinear dynamics and ever-changing disturbances. In this paper, a supplementary controller based on reinforcement learning (RL) is proposed to improve the control performance of quadrotor UAVs. The proposed RL method is constructed by an actor-critic structure and some improved technologies, e.g., Q-learning, temporal difference, and experience replay. With the proposed method, the speed and stability of training can be improved greatly. On one hand, the supplementary controller can work together with the traditional controller online, which can guarantee the stability of the system. On the other hand, the model uncertainties and external disturbances could be restrained through online RL training. The Lyapunov theory is used to prove the convergence of the RL controller's weights theoretically. Finally, three simulations are provided to illustrate the effectiveness of the proposed controller.