Reinforcement Learning for Distributed Energy Efficiency Optimization in Underwater Acoustic Communication Networks

Abstract

To solve the problems of poor quality of service and low energy efficiency of nodes in underwater multinode communication networks, a distributed power allocation algorithm based on reinforcement learning is proposed. The transmitter with reinforcement learning capability can select the power level autonomously to achieve the goal of getting higher user experience quality with lower power consumption. Firstly, we propose a distributed power optimization model based on the Markov decision process. Secondly, we further give a reward function suitable for multiobjective optimization. Finally, we present a distributed power allocation algorithm based on Q-learning and use it as an adaptive mechanism to enable each transmitter in the network to adjust the transmit power according to its own environment. The simulation results show that the proposed algorithm not only increases the total channel capacity of the system but also improves the energy efficiency of each transmitter.