Target Search Control of AUV in Underwater Environment with Deep Reinforcement Learning

Abstract

The autonomous underwater vehicle (AUV) is widely used to search for unknown targets in the complex underwater environment. Due to the unpredictability of the underwater environment, this paper combines the traditional frontier exploration method with deep reinforcement learning (DRL) to enable the AUV to explore the unknown underwater environment autonomously. In this paper, a grid map of the search environment is built by the grid method. The designed asynchronous advantage actor-critic (A3C) network structure is used in the traditional frontier exploration method for target search tasks. This network structure enables the AUV to learn from its own experience and generate search strategies for the various unknown environment. At the same time, DRL and dual-stream Q-learning algorithms are used for AUV navigation to further optimize the search path. The simulations and experiments in an unknown underwater environment with different layouts show that the proposed algorithm can accomplish target search tasks with a high success rate, and it can adapt to different environments. In addition, compared to other search methods, the frontier exploration algorithm based on DRL can search a wider environment faster, which results in a higher search efficiency and reduced search time.