Evaluating the Effectiveness of Protection Jamming Devices in Mitigating Smart Speaker Eavesdropping Attacks Using Gaussian White Noise

Abstract

Protection Jamming Devices (PJD) are specialized tools designed to sit on top of virtual assistant (VA) smart speakers and hinder them from “hearing” nearby user speech. PJDs aim to protect you from eavesdropping attacks by injecting a jamming signal directly into the microphones of the smart speaker. However, current signal processing routines can be used to reduce noise and enhance speech contained in noisy audio samples. Therefore, we identify a potential vulnerability for speech eavesdropping via smart speaker recordings, even when a PJD is being used. If an attacker can gain access to or facilitate smart speaker recordings they may be able to compromise a user's speech with successful noise cancellation. Specifically, we are interested in the potential for Gaussian white noise (GWN) to be an effective jamming signal for a PJD. To our knowledge, the effectiveness of white noise and PJDs to protect against eavesdropping attacks has yet to receive a systematic evaluation that includes physical experiments with an actual PJD implementation. In this work we construct our own PJD, specialized for consistent experimentation, to simulate an attack scenario where recordings from a smart speaker, in the presence of normal speech and the PJDs jamming signal, are recovered. We perform substantial data collection under different settings to build a repository of 1500 recovered audio samples. We applied post-processing on our dataset and conducted an extensive signal/speech quality analysis including both time and frequency domain inspection, and evaluation of metrics including cross-correlation, SNR, and PESQ. Lastly, we performed feature extraction (MFCC) and built machine learning classifiers for tasks including speech (digit) recognition, speaker identification, and gender recognition. We also attempted song recognition using the Shazam app. For all speech recognition tasks that we attempted, we were able to achieve classification accuracies above that of random guessing (46% for digit recognition, 51% for speaker identification, 80% for gender identification), as well as demonstrate successful song recognition. These results highlight the real potential for attackers to compromise user speech, to some extent, using smart speaker recordings; even if the smart speaker is protected by a PJD.