Automated quality monitoring for call centers using speech and NLP technologies

Abstract

This paper describes an automated system for assigning quality scores to recorded call center conversations. The system combines speech recognition, pattern matching, and maximum entropy classification to rank calls according to their measured quality. Calls at both ends of the spectrum are flagged as “interesting” and made available for further human monitoring. In this process, the ASR transcript is used to answer a set of standard quality control questions such as “did the agent use courteous words and phrases,” and to generate a question-based score. This is interpolated with the probability of a call being “bad,” as determined by maximum entropy operating on a set of ASR-derived features such as “maximum silence length” and the occurrence of selected n-gram word sequences. The system is trained on a set of calls with associated manual evaluation forms. We present precision and recall results from IBM’s North American Help Desk indicating that for a given amount of listening effort, this system triples the number of bad calls that are identified, over the current policy of randomly sampling calls. The application that will be demonstrated is a research prototype that was built in conjunction with IBM’s North American call centers.