Attack Some while Protecting Others: Selective Attack Strategies for Attacking and Protecting Multiple Concepts

Abstract

Machine learning models are vulnerable to adversarial attacks. Existing research focuses on attack-only scenarios. In practice, one dataset may be used for learning different concepts, and the attacker may be incentivized to attack some concepts but protect the others. For example, the attacker might tamper a profile image for the “age” model to predict “young”, while the “attractiveness” model still predicts “pretty”. In this work, we empirically demonstrate that attacking the classifier for one learning task may negatively impact classifiers learning other tasks on the same data. This raises an interesting research question: is it possible to attack one set of classifiers while protecting the others trained on the same data? Answers to the above question have interesting implications for the complexity of test-time attacks against learning models, such as avoiding the violation of logical constraints. For example, attacks on images of high school students should not cause these images to be classified as a group of 30-year-old. Such misclassification of age may raise alarms and may easily expose the attacks. In this paper, we address the research question by developing novel attack techniques that can simultaneously attack one set of learning models while protecting the other. In the case of linear classifiers, we provide a theoretical framework for finding an optimal solution to generating such adversarial examples. Using this theoretical framework, we develop a “multi-concept” attack strategy in the context of deep learning tasks. Our results demonstrate that our techniques can successfully attack the target classes while protecting the “protected” classes in many different settings, which is not possible with the existing test-time attack-only strategies.