Visemes of Chinese Shaanxi xi’an dialect talking head

Abstract

Animated 3D articulation models – called talking heads – can be utilized, for instance, in speech assistant systems for children who are hard-of-hearing or when teaching learners of a second language. In this study, the objective is to identify articulation features and a dynamic system for visual representation of speech sounds for a Shaanxi Xi’an dialect talking head. In the first phase of the study, a phonetic alphabet of the dialect (northwest China) is formed following the official Romanization system used for Mandarin (Standard Chinese). After relating the phonemes of the dialect to those of Mandarin, we introduce the SAMPA code developed for the dialect, in addition to the correspondent regularities for whole syllable pronunciation. Secondly, we display the classification of static visemes (phonemes represented in visual form) for the dialect and describe an experiment carried out to articulatory movements of the tongue (features of timing and position) in dialect speech utterances recorded at different tempos. Finally, we discuss the results of an analysis of the images based on spatial-temporal tracking of the tongue movement contour. For definition of each uttered viseme the visual information obtained is classified and then used to create the dynamic viseme system of the tongue for a talking head using the Shaanxi Xi’an dialect of Chinese.