Augmented reality anatomy visualization for surgery assistance with hololens: AR surgery assistance with hololens

Abstract

Immediate care for trauma wounded patients in austere or remote settings makes medical knowledge, skills, and efficiency of the on-duty medical professional paramount. For wounds that extend deep into internal anatomy, proper visualization of internal anatomy can enable more efficient and effective evaluation when presented to medical providers positioned close to the point of injury (POI). In this paper, a conceptual Augmented Reality (AR) surgical tool is presented to provide visualization of internal human anatomy, superimposed on the view of a patient, to assist medical providers for immediate casualty care. This AR surgical tool can play a role in 3D surgery or treatment planning as a navigational aid in preparing medical interventions and enhancing surgery or treatment procedures by displaying otherwise obscured anatomy and nearby vessels. Critical software and hardware components are integrated to construct a prototype AR system for the portable AR surgical visualization tool. The system uses a Microsoft HoloLens 1 and an Azure Kinect camera for simultaneous body-tracking and anatomy overlay to demonstrate the overall concept. Future extension of this work will aim to create a more accurate and compact prototype system that utilizes HoloLens 2 with an embedded Kinect camera for laboratory and field tests of its use in surgery assistance. Such an AR tool can also serve as a training tool for medical caregivers, applied with a human subject or a medical manikin.