Deep Reinforcement Learning-Based UAV Path Planning for Energy-Efficient Multitier Cooperative Computing in Wireless Sensor Networks

Abstract

Benefiting from the progress of microelectromechanical system (MEMS) technology, wireless sensor networks (WSNs) can run a large number of complex applications. One of the most critical challenges for complex WSN applications is the huge computing demands and limited battery energy without any replenishment. The recent development of UAV-assisted cooperative computing technology provides a promising solution to overcome these shortcomings. This paper addresses a three-tier WSN model for UAV-assisted cooperative computing, which includes several sensor nodes, a moving UAV equipped with computing resources, and a sink node (SN). Computation tasks arrive randomly at each sensor node, and the UAV moves around above the sensor nodes and provides computing services. The sensor nodes can process the computation tasks locally or cooperate with the UAV or SN for computing. In a life cycle of the UAV, we aim to maximize the energy efficiency of cooperative computing by optimizing the UAV path planning on the constraints of node energy consumption and task deadline. To adapt to the time-varying indeterminate environment, a deep Q network- (DQN-) based path planning algorithm is proposed. Simulation studies show that the performance of the proposed algorithm is better than the competitive algorithms, significantly improves the energy efficiency of cooperative computing, and achieves energy consumption balance.