A wearable multisensory, multiagent approach for detection and mitigation of acute cognitive strain: Phase I - Vocalization analysis

1Citations
Citations of this article
17Readers
Mendeley users who have this article in their library.

This article is free to access.

Abstract

While operators performing tasks with high workload can increase task performance in response to limited increases in cognitive stress, chronic or rapidly accelerating stress can exceed the operator’s ability to compensate, generating acute cognitive strain (ACS). ACS represents a state wherein performance, situation awareness and cooperativity deteriorate markedly, leading to critical errors, mishaps or casualties. Nearly two decades of augmented cognition (AugCog) research has demonstrated the utility of psychophysiologic sensing and analysis for identification and tracking of changes in cognitive state and to modulate human machine interactions for improving system task performance. The proposed approach leveraged prior efforts to modulate cognitive stress using a multiagent approach to acquire and analyze multiple Psychophysiologic sensory channels, including changes in vocalizations, to create a reliable and non-intrusive Detector of Acute Cognitive Strain (DACS). The DACS system provides an integrated wearable multi-modal Research Sensor Suite (RSS) using the open-source Adaptive Multiagent Integration (AMI) architecture, that includes analysis agents for electroencephalograph (EEG), electromyography (EMG), video oculography (VOG), vocalization, and others to identify and correlate physiological signatures with cognitive stress and strain. An online AMI agent-based processing algorithm was developed and applied to audio communications to evaluate for changes in speaker vocalization fundamental frequency (F0) and cadence (utterances per minute). This paper describes initial phase results of aerospace mishap vocalization stress marker detection, a potential element of the proposed DACS system. DACS could use these markers to trigger adaptive automation agents that reduce task load and allow pilots to prevent or recover from ACS episodes.

Cite

CITATION STYLE

APA

Raj, A., Roberts, B., Hollingshead, K., McDonald, N., Poquette, M., & Soussou, W. (2018). A wearable multisensory, multiagent approach for detection and mitigation of acute cognitive strain: Phase I - Vocalization analysis. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Vol. 10915 LNAI, pp. 180–200). Springer Verlag. https://doi.org/10.1007/978-3-319-91470-1_16

Register to see more suggestions

Mendeley helps you to discover research relevant for your work.

Already have an account?

Save time finding and organizing research with Mendeley

Sign up for free