An Improved Reinforcement Learning Approach for Sustainable 6G UAV Communications

Abstract

The sixth-generation communications networks (6G) are expected to be deployed in the 2030s with integrated space-aerial-ground and undersea architecture to provide seamless global connectivity. In this architecture, unmanned aerial vehicles (UAVs) are one of the most unique characteristics and are becoming increasingly important. The flexibility, high speed, and infrastructure independence of UAV systems make them ideal for many applications. However, these advantages also create great challenges in effective communication between UAVs. To address these challenges, reinforcement learning (RL) algorithms such as Q-Learning have been investigated. However, the traditional Q-learning algorithm mainly relies on delay parameters in the reward function for decision-making route selection. Aiming to optimise the selection of sustainable and efficient communication for UAVs, we propose an improved routing algorithm based on Q-Learning for UAV communication. Our method integrates latency, energy consumption, and link quality parameters into the reward function to make smarter routing decisions. The simulation results show that Q-Proposed achieves significant gains in terms of packet delivery ratio and end-to-end delay compared to other methods, paving the way for sustainable 6G UAV communications.