Hand signal recognition for handling surgical instruments

Abstract

The provision of surgical instruments is often done by surgeon and nurse clinicians but this way sometimes encounters some issues such as supplying the wrong instruments, delays, or seriously that is to forget the surgical instruments inside the body of a patient. This study presents a computer vision system which identifies the hand symbols and the surgical instruments’ position using a Kinect sensor in the operating room. The system aims at finding and locating a hand and its midpoint based on a depth image from Kinect sensor and it is thus able to recognize the hand symbols from a surgeon in order to control a manipulator to pick the right instrument. Then, this system detects position and direction of this instrument on a table to direct a robot to pick it up properly. The experiments were conducted to show the positive results of this system.