Patient adapted augmented reality system for real-time echocardiographic applications

Abstract

When compared to other imaging modalities, the acquisition and interpretation of ultrasound data is challenging for new users. The main aim of this work is to investigate whether augmented reality can provide a patient specific correspondence between the ultrasound image and a virtual anatomic model of the heart and thus simplify the live echocardiographic scanning process. The proposed augmented view contains the image of the scene (acquired with a tablet’s camera), a generic heart model and the ultrasound image plane. The position and orientation of the ultrasound probe and patient are tracked with the Vuforia framework, using fiducial markers that are visible on the tablet’s camera image. A customized renderer for augmentation purposes was implemented in OpenGL ES. Data streaming from a high end ultrasound scanner to the tablet device was implemented and the position of the heart model is continuously updated to match the ultrasound data by tracking and matching a set of anatomic landmarks. The prototype was tested in the echo laboratory and realtime performance was achievable as no significant lag between the scanner image and the one presented on the tablet was experienced. Furthermore the suitability of Vuforia for fiducial tracking was evaluated and was deemed sufficiently accurate for this application. The presented prototype was tested by an experienced echocardiographer and was considered beneficial for both teaching purposes and as a help tool for inexperienced ultrasound users.