On-Device System for Device Directed Speech Detection for Improving Human Computer Interaction

Abstract

Voice assistants (VA) are finding a place in many households, with increasing numbers. Nevertheless, for every interaction user invokes VA using a key or wake-up word, which is too common and hinders natural conversation. To solve this, we propose an On-Device solution that listens to the user continuously for only predefined period and classifies the utterance into device-directed or non-device-directed using a deep learning-based model. Since our solution is On-Device, the privacy of the user is maintained. We tried to solve false acceptance of utterance as a command and incorrect rejection of the command as background noise to improve user experience. We calculated the False Acceptance Rate (FAR) and False Rejection Rate (FRR) for different thresholds to evaluate our model. Then we calculated Equal Error Rate (EER) results with the FRR and FAR data. We achieved 3.6% of EER on our best optimized Stage-1 model and EER of 5.1% on Stage-2 model. We have also used accuracy as evaluation metrics during training and testing. We achieved an accuracy of 96.8% on testing data with our Stage-1 model and 95.3% on Stage-2 model.