Interactive gestures for liver Angiography operation

Abstract

The main challenge of creating large interactive displays in the operating rooms (ORs) is in the definition of ways that are efficient and easy to learn for the physician. Apart from traditional input methods such as mouse and keyboard, we have developed a multimodal system with two different vision based human-computer interaction (HCI) systems that can simplify the way surgeons interact with the medical images shown on the LCD display. The purpose of this work is to construct a gesture recognition system with a fast, accurate, and easily attainable method. The first system is a laser pointer interaction framework that supports a 2D stroke gesture interface. The recorded laser gestures are recognized using two different algorithms: dynamic time warping (DTW) and one dollar (1$) recognizer. Our experimental results showed that the DTW algorithm performs better with an overall accuracy of 90 %. The second prototype presents an intuitive HCI to manipulate images using freehand gestures. In order to strengthen the gesture recognition process, the system incorporates contextual information to determine the intent of the user of interacting with the large display. Two cameras are used to observe the surgeon’s hand movements to continuously determine and monitor what the surgeon intends to perform. Experimental results showed that the system accuracy is 95 % for recognition with the effect of contextual integration.