Efficient On-Device Domain Learning for Keyword Spotting on Ultra-Low-Power Platforms

Abstract

Deep Neural Network (DNN)-based keyword spotting (KWS) accuracy degrades in noisy environments. On-site adaptation to previously unseen noise is crucial to recover accuracy loss, and On-Device Learning (ODL) is required in scenarios where adaptation has to happen in the field. In this work, we propose a fully On-Device Domain Adaptation (ODDA) system, enabling edge devices to achieve noise-robust KWS. We achieve up to 14% accuracy gains over already-robust KWS models and up to 21% increments when evaluating our methodology on keyword datasets disjoint from the offline training set. In extreme edge scenarios where KWS is critical, using as little as 10 kB of memory and only 100 labeled utterances, we enable ODL and demonstrate accuracy recovery of up to 5% after adapting to complex, non-stationary speech noise. We show that domain adaptation can be achieved on ultra-low-power (ULP) microcontrollers with as little as 357 mJ within 14 s on always-on, battery-operated devices. This work is the first to demonstrate an end-to-end ODDA system for noise-robust KWS models on ULP, extreme edge platforms.