Misactivation detection and user identification in smart home speakers using traffic flow features

Abstract

The advancement in Internet of Things (IoT) technology has transformed our daily lifestyle. Particularly, voice assistants such as Amazon's Alexa and Google Assistant are commonly deployed in households. These voice assistants enable users to interact with other devices in a smart home ecosystem. In this paper, we focus on two security issues that arise with the use of smart speakers, and present network flow fingerprinting methods to mitigate their impact. First, we concentrate on the misactivation of smart speakers in which spoken words unintentionally activate the device. This may lead to private user conversations being recorded and sent to the cloud without the user even noticing. To prevent such misactivation, we explore locality-sensitive hash-based machine learning approaches. Our evaluation results with the network traffic of four different smart speakers show that the proposed approach can achieve an area under the curve (i. e., AUC) of 93% to 99%. Secondly, we explore whether the voice commands of the device owner can be distinguished from other individuals based on the generated network traffic fingerprint without any analysis of the actual sound wave. Evaluation results with five different user voices show that we can achieve an average AUC of 72% to 81% by ensembling multiple machine learning models.