Deep reinforcement learning based online lifting path planning for tower cranes in unknown dynamic environments

Abstract

Lifting path planning is critical for the safety and efficiency of tower cranes operating in dynamic construction environments. This paper proposes a lifting path planner to efficiently generate safe and smooth lifting paths for tower cranes in an unknown construction environment through a deep reinforcement learning (DRL) method. Based on the Twin-Delayed DDPG (TD3) framework, the planner effectively plans a lifting path within constraints of collision avoidance and operational limitations using the local environmental information measured by lidar. A Long Short-Term Memory network is applied in the planner to handle the dynamic characteristics of the obstacles in the construction sites to ensure that the lifting path is collision-free with dynamic obstacles. A discrete-continuous hybrid action space for tower cranes is proposed to optimize planned lifting paths more suitable for practical engineering operations. Moreover, a novel reward function is introduced to optimize the smoothness of the lifting path, which improves the success rate and optimizes the energy and time cost. A new Hindsight Experience Replay algorithm is proposed to address the reward sparsity problem in lifting path planning, which improves the training speed. Simulation results in Webots platform show the presented method effectively reduces the planning time and achieves better performance on online path planning compared with the existing DRL path planning methods.