Trojan attack on deep generative models in autonomous driving

Abstract

Deep generative models (DGMs) have empowered unprecedented innovations in many application domains. However, their security has not been thoroughly assessed when deploying such models in practice, especially in those mission-critical tasks like autonomous driving. In this work, we draw attention to a new attack surface of DGMs, which is the data used in the training phase. We demonstrate that the training data poisoning, the injection of specially-crafted data, are able to teach Trojan behaviors to a DGM without influencing the original training goal. Such Trojan attack will be activated after model deployment only if certain rare triggers are present in an input. For example, a rain-removal DGM after poisoning can, while removing raindrops in input images, change a traffic light from red to green if this traffic light has a specific appearance (i.e. a trigger). Clearly severe consequences can occur if such poisoned model is deployed on vehicle. Our study shows that launching our Trojan attack is feasible on different DGM categories designed for the autonomous driving scenario, and existing defense methods cannot effectively defeat it. We also introduce a concealing technique to make our data poisoning more inconspicuous during the training. In the end, we propose some potential defense strategies inspiring future explorations.