Image overlay support with 3DCG organ model for robot-assisted laparoscopic partial nephrectomy

Abstract

We have developed a surgery support system for robot-assisted laparoscopic partial nephrectomy. In our system, a 3D computer graphics (3DCG) model, which includes a kidney, arteries, veins, and tumors, is saved separately and overlaid on the surgeon’s viewing endoscopic camera image through the operator’s console monitor in real time. The position and orientation of the 3DCG model is calculated from the optical flow of the endoscopic camera images, and the model is moved semi-automatically. The display condition of each part of the 3DCG model is independently controlled anytime by using a mouse, rotary controllers, or key board.