Deep learning approach to automated segmentation of tongue in camera images for computer-aided speech diagnosis

Abstract

This paper describes an approach for automated segmentation of tongue in camera images for computer-aided speech diagnosis and therapy. Speech disorders are often related to non-normative position of articulators. One of common pathologies in Polish pronunciation is interdentality, when the tongue protrudes between the front teeth. Segmentation and possible parametrization of tongue in camera images could support speech diagnosis. Presented system is based on images captured by two cameras directed at speaker’s mouth at different angles on the left and right side. A convolutional neural network was designed and trained for semantic segmentation of tongue. Three datasets of input data were examined, two taken from each camera separately and one combined from both cameras. The mean Jaccard index reached 74.01% over the combined dataset with the corresponding accuracy at 96.09%.