Impact of preprocessing features on the performance of ultrasound tongue contour tracking, via dynamic programming

Abstract

The automated assessment of ultrasound images for speech processing is a difficult process. The number of frames processed, amounts to several hundred thousand, this makes assessment nearly impossible to process manually. Tongue contour tracking is indispensable for the dynamic modelling of articulation. The difficulty of the task lies in the fact that the images have a noisy background and the contour curve is discontinuous. An algorithm based on dynamic programming has been developed to track the movement of the back of the tongue. With an extreme size edge enhancing and averaging construction, the procedure addresses the problems of break of discontinuity and noise, simultaneously. In the image obtained after smoothing, the brightest curve is sought, from the left to the right edges of each image. The points of the curve thus obtained, follow the uneven line of the tongue contour. To smooth the curve, filtering based on Discrete Cosine Transformation (DCT) is applied. With the appropriate selection of universal parameters and processing the signals of several speakers in an identical way, the accuracy of edge detection can be enhanced considerably. We have optimized and qualified the results, comparing them with manual contour tracking. The accuracy of contour tracking may be improved by applying speaker-specific adjustments. The results of this analysis define temporary data for articulation key frames for visual speech synthesis (a talking head). Beyond the static analysis, we also investigate the trajectories of articulation features over time. We refined our previously created dynamic model, in order to construct a full dataset for the articulation.