Q-learning-based routing inspired by adaptive flocking control for collaborative unmanned aerial vehicle swarms

Abstract

A collaborative unmanned aerial vehicle (UAV) swarm can support various applications, including aerial surveillance and emergency communication. Owing to the high mobility, limited energy of UAVs, and frequent link breakages, data routing from remote UAVs to a base station may produce retransmissions, loops, energy holes, and long delay. In a UAV swarm network, therefore, relative mobility, path stability, and delay should be jointly taken into consideration to improve routing performance because they are highly coupled with each other. However, they are not properly exploited in the existing literature. To address these issues, in this paper, a Q-learning (QL)-based routing protocol inspired by adaptive flocking control (QRIFC) is proposed. The proposed adaptive flocking control algorithm generates optimal mobility with fairness in travel distance for each UAV to control the optimal node density. It also addresses the trade-off between aerial coverage and quality of service in connectivity by imposing constraints on the minimum separation distance and maximum allowable inter-UAV spacing using two-hop neighbor information. Additionally, it provides a stable link duration (LD) between neighboring UAVs and optimizes the control overhead. Furthermore, QL performs multi-objective optimization by utilizing a new state exploration and exploitation strategy to select an optimal routing path in terms of delay, stable path selection defined by predictive LD, and energy consumption. According to an extensive performance study, the proposed QRIFC outperforms existing routing protocols by 21-40 percent of average end-to-end delay and 9-23 percent of average packet delivery ratio, with less retransmissions.