Automatic fast-registration surgical navigation system using depth camera and integral videography 3D image overlay

Abstract

We propose an automatic fast-registration augmented reality (AR) surgical navigation system for minimally invasive surgery. The system integrates a fast-registration technique with three-dimensional (3D) integral videography (IV) image overlay. The detailed anatomic information generated by IV technique is superimposed to the patient using 3D autostereoscopic images, which reproduce motion parallax with naked eyes. To reduce the patient-3D overlay image registration time and achieve automatic execution, we integrate a 3D image overlay system with a real-time patient tracking system, which utilizes particle filter algorithm and depth camera. Experimental results showed that the system can lower the registration time and the can reach up to 10 frames per second (fps). The 3D overlay image to the patient registration average error is 1.88 mm, with standard deviation of 0.72 mm. Further work for improvement of the depth camera acquisition accuracy and tracking algorithm makes this system more feasible and practical.