Cognitive considerations in auditory user interfaces: Neuroergonomic evaluation of synthetic speech comprehension

Abstract

Automated spoken language interfaces have seen a remarkable proliferation in recent years, integrating with automotive, household, industrial, and mobile platforms to shape the way in which we interact with our devices. While the use of an auxiliary auditory information stream has the potential to decrease interference and prevent disengagement from operation of traditional visual/mechanical interfaces, evidence from behavioral and neuroimaging studies have suggested that the brain mechanisms underlying the perception and comprehension of synthetic speech may be different from naturally produced speech, resulting in an unnecessary additional cognitive burden. In this neuroergonomics study, functional Near-Infrared Spectroscopy (fNIRS) over the anterior prefrontal cortex has been measured to determine the influence of synthetic speech quality during a sentence comprehension and quality assessment task. Eight participants were asked to listen to topical sentences from real-world audio interfaces employed in car driving scenarios and then answer questions regarding the content of the messages and rate the quality (Intelligibility and Naturalness) of the audio. Results indicate that the behavioral performance during assessment of speech content and rated Intelligibility were negatively impacted when using lower quality synthetic voices. Performance costs associated with low-quality synthetic voices were related to increased cognitive load as measured by increased medial prefrontal cortex activity. Approaches and concepts described here can be used to guide next-gen speech synthesizer design and future research for decreasing the cognitive load in driving scenarios.