Using Augmented Reality and Machine Learning in Radiology

Abstract

Surgeries are one of the main cost factors of health care systems. To reduce the costs related to diagnoses and surgeries, we propose a system for automated segmentation of medical images in order to segment body parts like liver or lesions. The model is based on convolutional neural networks, for which we show promising results on real computed tomography scans. The deep learning algorithm is part of a larger system that aims to support doctors by visualizing the segments in a Microsoft HoloLens, an augmented reality device. Our approach allows doctors to intuitively look at and interact with the holographic data rather than using 2D screens, enabling them to provide better health care. Both the machine learning algorithm and the visualization utilize high-performance GPUs in order to enable doctors to interact efficiently with our system.