Predicting Customer Satisfaction with Soft Labels for Ordinal Classifcation

Abstract

In a typical call center, only up to 8% of callers leave a Customer Satisfaction (CSAT) survey response at the end of the call, and these tend to be customers with strongly positive or negative experiences. To manage this data sparsity and response bias, we outline a predictive CSAT deep learning algorithm that infers CSAT on the 1-5 scale on inbound calls to the call center with minimal latency. The key metric to maximize is the precision for CSAT = 1 (lowest CSAT). We maximize this metric in two ways. First, reframing the problem as a binary class, rather than fve-class problem during model fne-tuning, and then mapping binary outcomes back to fve classes using temperature-scaled model probabilities. Second, using soft labels to represent the classes. The result is a production model that supports key customer work-fows with high accuracy over millions of calls a month.