Autonomous Obstacle Avoidance with Improved Deep Reinforcement Learning Based on Dynamic Huber Loss

Abstract

In dynamic and unstructured environments, the obstacle avoidance capabilities of Unmanned Aerial Vehicles (UAVs) are crucial for mission success. Traditional methods struggle with adaptability and effectiveness in unknown or changing scenes. In contrast, the commonly used deep reinforcement learning (DRL) ones suffer from slow convergence, reduced accuracy, and inadequate robustness due to the loss of sensitivity to outliers and parameter rigidity. To address these challenges, we propose an enhanced DRL framework that leverages a Dynamic Huber loss function tailored for UAV autonomous obstacle avoidance. By incorporating Soft updates for target network and dynamically tuning the Huber loss, the proposed method facilitates faster model convergence, superior control precision, and improved robustness. Both theoretical analysis and experimental simulation verify its effectiveness with superior planning success rate, shorter average path length, and faster model convergence over traditional approaches. Specifically, in static environments, the Dynamic Huber-loss-based DRL framework achieves a 98.85% success rate with an optimized average path length of 10.73; in dynamic environments, it attains a 74.20% success rate with an average path length of 37.04; adding wind disturbances in a dynamic environment, it attains a 70.95% success rate with an average path length of 40.40, highlighting its enhanced performance and adaptability.