MR Device-Based Remote Medical Support System With Object Recognition

Abstract

The field of digital healthcare is rapidly evolving due to advancements in artificial intelligence technologies, particularly deep learning. This study aims to explore new possibilities in telemedicine by applying mixed reality (MR) technology to emergency medical situations. The primary goal of this research is to develop an MR device-based remote medical support system that utilizes MR devices and high-speed wireless communication to transmit onsite conditions to medical control rooms, enabling emergency responders to provide treatment under remote guidance. The system leverages Microsoft's HoloLens 2 to capture and transmit patient biometric information and surrounding environments, utilizing deep learning algorithms for human skeleton recognition and object alignment. The generated 3D avatars and content are then used in remote medical consultations. This article highlights the potential medical applications of HoloLens 2, a cutting-edge consumer electronics device, demonstrating how this system can provide innovative medical solutions through consumer technology and open up new possibilities in the healthcare field.