Deep Reinforcement Learning-Based Robotic Puncturing Path Planning of Flexible Needle

Abstract

The path planning of flexible needles in robotic puncturing presents challenges such as limited model accuracy and poor real-time performance, which affect both efficiency and accuracy in complex medical scenarios. To address these issues, this paper proposes a deep reinforcement learning-based path planning method for flexible needles in robotic puncturing. Firstly, we introduce a unicycle model to describe needle motion and design a hierarchical model to simulate layered tissue interactions with the needle. The forces exerted by tissues at different positions on the flexible needle are considered, achieving a combination of kinematic and mechanical models. Secondly, a deep reinforcement learning framework is built, integrating obstacle avoidance and target attraction to optimize path planning. The design of state features, the action space, and the reward function is tailored to enhance the model’s decision-making capabilities. Moreover, we incorporate a retraction mechanism to bolster the system’s adaptability and robustness in the dynamic context of surgical procedures. Finally, laparotomy simulation results validate the proposed method’s effectiveness and generalizability, demonstrating its superiority over current state-of-the-art techniques in robotic puncturing.