Deep learning has recently become the state of the art in many com-puter vision applications and in image classification in particular. However, recent works have shown that it is quite easy to create adversarial examples, i.e., images intentionally created or modified to cause the deep neural network to make a mistake. They are like optical illusions for machines containing changes unnoticeable to the human eye. This represents a serious threat for machine learn-ing methods. In this paper, we investigate the robustness of the representations learned by the fooled neural network, analyzing the activations of its hidden layers. Specifically, we tested scoring approaches used for kNN classification, in order to distinguishing between correctly classified authentic images and adversarial ex-amples. The results show that hidden layers activations can be used to detect incorrect classifications caused by adversarial attacks.
Carrara, F., Falchi, F., Caldelli, R., Amato, G., Fumarola, R., & Becarelli, R. (2017). Detecting adversarial example attacks to deep neural networks. In Proceedings of the 15th International Workshop on Content-Based Multimedia Indexing - CBMI ’17 (pp. 1–7). New York, New York, USA: ACM Press. https://doi.org/10.1145/3095713.3095753