A Multimodal Educational Game for 3-10-Year-Old Children: Collecting and Automatically Recognising European Portuguese Children’s Speech

Abstract

Speech interfaces have tremendous potential in education. In this paper, we present our work in the Contents for Next Generation Networks project, an ongoing Portuguese industry-academia collaboration developing a multimodal educational game aimed at improving the physical coordination and the basic mathematical and musical skills of 3-10-year-old children. We focus on our work in the area of children’s speech recognition: designing, collecting, transcribing and annotating a 21-hour corpus of prompted European Portuguese children’s speech, as well as our first experiments with different acoustic modelling approaches. Our speech recognition results suggest that training children’s speech models from scratch is a more promising approach than retraining adult speech models using children’s speech when a sufficient amount of training data is available from the targeted age group. This finding also holds for adult female speech models retrained using children’s speech. As compared with a baseline recogniser comprising gender-dependent adult speech models, the best-performing children’s speech models that we have trained so far – gender-independent cross-word triphones trained with 17.5 hours of speech from 3-10-year-old children – resulted in a 45-percent (relative) decrease in word error rate in a task expecting isolated cardinal numbers, sequences of cardinal numbers or musical notes as speech input.