Design and Analysis of Adversarial Samples in Safety–Critical Environment: Disease Prediction System

Abstract

Deep learning is a part of machine learning applied in many applications, from object detection to disease prediction. In 2013, deep neural networks were vulnerable to perturbed samples during the testing/deployment stage. Deep learning becomes a significant issue when we apply such algorithms in designing safety–critical applications. Deep neural networks are fooled by adding some noise intentionally in the image such that the changes are sometimes not noticeable to the human eye and are known as an adversarial mechanism. The noise adds in such a direction that the difference between the original and perturbed images should be minimal. This work demonstrates the designing of adversarial images and analyzing them on the disease prediction using a chest X-ray prototype system. This work illustrates the design of adversarial samples using three different methods: an image augmentation scheme, filtering, and patches and analysis of how adversarial samples affect the result in real-world clinical settings. The change in the clinical setting not only affects the healthcare economy but also raises technical vulnerability. This work attempts to design and improve a more robust medical learning system. The analysis shows if patches applied in specific regions give expected results to the adversary. In this work, after the successful attack model shows 71% of confidence.