Pteacher: A computer-aided personalized pronunciation training system with exaggerated audio-visual corrective feedback

Abstract

Second language (L2) English learners often fnd it difcult to improve their pronunciations due to the lack of expressive and personalized corrective feedback. In this paper, we present Pronunciation Teacher (PTeacher), a Computer-Aided Pronunciation Training (CAPT) system that provides personalized exaggerated audio-visual corrective feedback for mispronunciations. Though the efectiveness of exaggerated feedback has been demonstrated, it is still unclear how to defne the appropriate degrees of exaggeration when interacting with individual learners. To fll in this gap, we interview 100 L2 English learners and 22 professional native teachers to understand their needs and experiences. Three critical metrics are proposed for both learners and teachers to identify the best exaggeration levels in both audio and visual modalities. Additionally, we incorporate the personalized dynamic feedback mechanism given the English profciency of learners. Based on the obtained insights, a comprehensive interactive pronunciation training course is designed to help L2 learners rectify mispronunciations in a more perceptible, understandable, and discriminative manner. Extensive user studies demonstrate that our system signifcantly promotes the learners' learning efciency.