Straight-Path Following and Formation Control of USVs Using Distributed Deep Reinforcement Learning and Adaptive Neural Network

Abstract

Dear Editor, This letter presents a distributed deep reinforcement learning (DRL) based approach to deal with the path following and formation control problems for underactuated unmanned surface vehicles (USVs). By constructing two independent actor-critic architectures, the deep deterministic policy gradient (DDPG) method is proposed to determine the desired heading and speed command for each USV. We consider the realistic dynamical model and the input saturation problem. The radial basis function neural networks (RBF NNs) are employed to approximate the hydrodynamics and unknown external disturbances of USVs. Simulation results show that our proposed method can achieve high-level tracking control accuracy while keeping a desired stable formation.