FenceSitter: Black-box, Content-Agnostic, and Synchronization-Free Enrollment-Phase Attacks on Speaker Recognition Systems

Abstract

Speaker Recognition Systems (SRSs) grant access to legitimate users based on voiceprint. Recent research has shown that SRSs can be bypassed during the training phase (backdoor attacks) and the recognition phase (evasion attacks). In this paper, we explore a new attack surface of SRSs by presenting an enrollment-phase attack paradigm, named FenceSitter, where the adversary poisons the SRS using imperceptible adversarial ambient sound when the legitimate user registers into the SRS. The tainted voiceprint extracted by the SRS allows both the adversary and the legitimate user to access the system in all future recognition phases. To materialize such attack, we interleave carefully-designed continuous adversarial perturbations into innocent-sounding ambient sound. As computing adversarial perturbations over a long sequence of ambient sound carrier is intractable, we optimize over adversarial segments with content desensitization and physical realization. In addition, the attack is made available under the black-box settings by gradient estimation based on the natural evolution strategy. Extensive experiments have been conducted on both English and Chinese voice datasets for close-set identification (CSI), open-set identification (OSI), and speaker verification (SV) tasks. The results under various digital and physical conditions have verified the effectiveness and robustness of FenceSitter. With live enrollment experiments and user study, we further validate the practicality of FenceSitter. Our work reveals the vulnerability of SRSs during the enrollment phase, which may spur future research in improving the security of SRSs.