Multidimensional real-time assessment of user state and performance to trigger dynamic system adaptation

Abstract

In adaptive human-machine interaction, technical systems adapt their behavior to the current state of the human operator to mitigate critical user states and performance decrements. While many researchers use measures of workload as triggers for adjusting levels of automation, we have proposed a more holistic approach to adaptive system design that includes a multidimensional assessment of user state. This paper outlines the design requirements, conceptual framework, and proof-of-concept implementation of a Real-time Assessment of Multidimensional User State (RASMUS). RASMUS diagnostics provide information on user performance, potentially critical user states, and their related impact factors on a second-by-second-basis in real-time. Based on these diagnoses adaptive systems are enabled to infer when the user needs support and to dynamically select and apply an appropriate adaptation strategy for a given situation. While the conceptual framework is generic, the implementation has been applied to an air surveillance task, providing real-time diagnoses for high workload, passive task-related fatigue and incorrect attentional focus.