Do the Urgent Things first! - Detecting Urgency in Spoken Utterances based on Acoustic Features

Abstract

In the future, spoken dialogue systems will have to deal with more complex user utterances and should react in an intuitive, comprehensible way by adapting to the user, the situation and the context. In rapidly changing situations, like talking to a highly automated car, it is highly relevant to react adequately to quick urgent interjections whether within one utterance or as interruptions of ongoing actions/dialogues. A first step is the detection of urgency in user utterances. Therefore, we developed a user study based on gamification simulating such short-term urgent situations. With this study, we collected data for a first analysis of features from the audio signal, which are promising for detecting urgent utterances. In the game "What is it?" participants had to find a symbol consisting of three characteristics from a set via speech. Their search was regularly interrupted by a time limited urgent task. The data obtained show that features only from the audio signal can be used to distinguish between urgent and non-urgent utterances. Further analysis reveals that certain features of the audio signal represent different phases of the data set better or worse. We distinguish, among other things, between the phases Transition and Decline, which represent the shift from non-urgent to urgent speech and vice versa. These shifts are recognizable and can occur in rapid change. We identified several classification methods to detect successfully urgent speech in each phase.