Calibration of online situation awareness assessment systems using virtual reality

Abstract

In an attempt to predict and prevent accident situations in complex socio-technical systems, one needs to be able to model and simulate concepts such as situation awareness (SA) and processes responsible for maintaining it. This is particularly true in the case of online support systems and adaptive displays which cannot rely on SA measurement techniques based on freeze probe techniques. This work investigates the state of the art in computational models of situation awareness and proposes a method to calibrate and evaluate such models using virtual reality human-in-the-loop experiments. This work introduces a new methodology to evaluate and calibrate online SA assessment systems taking advantage of the flexibility and reconfigurable power of virtual reality environments. This technology provides the experimenter with full control on the scenarios, cockpit types and interfaces. It also allows testing of off-nominal situations such as the loss of an instrument and more severe failures. Moreover, eye tracking capabilities provide an accurate way of registering monitoring events and feed SA assessment models with realistic data. © 2014 Springer International Publishing Switzerland.