Assessment and evaluation of speech-based interactive systems: From manual annotation to automatic usability evaluation

Abstract

Due to the improvements of speech and language technologies during the last few decades, the demand for assessment and evaluation of such technologies increased significantly. Starting from the assessment of individual system components such as automatic speech recognition (ASR) or text-to-speech (TTS) synthesis, evaluation methods are now required to address the system - and the service which is based on it - as a whole. Both individual component assessment and entire system evaluation are worth being considered here, depending on the question which shall be answered by the assessment or evaluation. © 2010 Springer Science+Business Media, LLC.