Exploring Extended Reality as a Simulation Training Tool Through Naturalistic Interactions and Enhanced Immersion

Abstract

Augmented reality (AR), mixed reality (MR), and virtual reality (VR) technologies – collectively extended reality (XR) – have been discussed in the context of training applications since their introduction to the market. While this discussion was initially founded on potential benefit, the rapid pace and advancements within the XR industry allow the technology to fulfill initial expectations in terms of training utility. As XR becomes an effective training tool, two open areas of research, among many, include integrating experiences between VR and AR/MR simulations, as XR training often obscures the trainer from the trainee’s environment, and determining the optimal level of fidelity and overall usability of interaction within XR simulations for the training of fine- and gross-motor control tasks, as new peripherals and technologies enter the market supporting a variety of interaction fidelities, such as data glove controllers, skeletal motion capture suits, tracking systems, and haptic devices. XR may show more promise as a training platform the closer it can replicate real world and naturalistic training interactions and immersion. This paper discusses efforts in these two research areas, including a planned usability study on the impact of the fidelity of interactions on training of fine- and gross-motor control tasks in virtual environments.