Collision-Free Robot Path Planning by Integrating DRL with Noise Layers and MPC

Abstract

With the rapid advancement of Autonomous Mobile Robots (AMRs) in industrial automation and intelligent logistics, achieving efficient and safe path planning in dynamic environments has become a critical challenge. These environments require robots to perceive complex scenarios and adapt their motion strategies accordingly, often under real-time constraints. Existing methods frequently struggle to balance efficiency, responsiveness, and safety, especially in the presence of continuously changing dynamic obstacles. While Model Predictive Control (MPC) and Deep Reinforcement Learning (DRL) have each shown promise in this domain, they also face limitations when applied individually—such as high computational demands or insufficient environmental exploration. To address these challenges, we propose a hybrid path planning framework that integrates an optimized DRL algorithm with MPC. We replace the Actor’s output with a learnable noisy linear layer whose mean and scale parameters are optimized jointly with the policy via backpropagation, thereby enhancing exploration while preserving training stability. TD3 produces stepwise control commands that evolve into a short-horizon reference trajectory, while MPC refines this trajectory through constraint-aware optimization to ensure timely obstacle avoidance. This complementary process combines TD3′s learning-based adaptability with MPC’s reliable local feasibility. Extensive experiments conducted in environments with varying obstacle dynamics and densities demonstrate that the proposed method significantly improves obstacle avoidance success rate, trajectory smoothness, and path accuracy compared to traditional MPC, standalone DRL, and other hybrid approaches, offering a robust and efficient solution for autonomous navigation in complex scenarios.