Recognition of Japanese attitudes in audio-visual speech

Abstract

The aim of the present work is to investigate how Japanese listeners recognize 12 audio-visual prosodic attitudes of Japanese. Significant influences of the two speakers and three modalities were observed. Generally the audio-visual condition showed the best recognition score, and interesting behavior for audio and visual modality was observed. Attitudes were regrouped into 3 higher-level perceptual categories for the first speaker: polite expressions, the attitudes of "query", and the expressions of imposition of one's own opinion. The attitude of kyoshuku and surprise are particularly well recognized by visual information.