Collaborative Multi-AUV Optical Communication via Deep Reinforcement Learning

Abstract

The next generation of wireless network 6G is envisioned to enable seamless global connectivity for the Internet of Everything (IoE) on the Earth. To this end, implementing high-speed wireless communication from the deep ocean to the sea surface leveraging multiple autonomous underwater vehicles (AUVs) with underwater wireless optical communication (UWOC) is an emerging and promising technology that enables real-time data collection for accurate underwater exploration and monitoring, for example, coordinated moving target monitoring. However, multihop UWOC is more susceptible to beam misalignment and positional uncertainty caused by external interference in the harsh environment. To address these challenges, we design a cooperative movement scheme for multiple AUVs based on a deep reinforcement learning (DRL) approach to perform robust optical communication. We first model the optical channel and then analyze the link performance to satisfy the bit error rate (BER) requirements. Afterward, we map the cooperative optical communication problem into a Markov decision process (MDP) and then we propose a deep deterministic policy gradient (DDPG)-based cooperative movement strategy, which is integrated with the extended Kalman filter (EKF) technique. Finally, we design a multi-AUV adaptive adjustment scheme for enhanced optical link adaptation, including an optical link distance adjustment algorithm, and an adaptive transmit power adjustment algorithm based on twin-delayed deep deterministic policy gradient (TD3). Through extensive simulations, it is demonstrated that the proposed algorithms are effective in achieving cooperative and adaptive underwater optical communication via multi-AUV under mobile scenarios.