Exploring Adversarial Examples: Patterns of One-Pixel Attacks
This work addresses adversarial vulnerabilities in healthcare AI, but it is incremental as it focuses on a simplified model rather than real-world medical images.
The study tackled the problem of adversarial examples in deep learning by simplifying a medical pose estimation task to analyze one-pixel attacks, finding that these attacks frequently succeed in specific image regions.
Failure cases of black-box deep learning, e.g. adversarial examples, might have severe consequences in healthcare. Yet such failures are mostly studied in the context of real-world images with calibrated attacks. To demystify the adversarial examples, rigorous studies need to be designed. Unfortunately, complexity of the medical images hinders such study design directly from the medical images. We hypothesize that adversarial examples might result from the incorrect mapping of image space to the low dimensional generation manifold by deep networks. To test the hypothesis, we simplify a complex medical problem namely pose estimation of surgical tools into its barest form. An analytical decision boundary and exhaustive search of the one-pixel attack across multiple image dimensions let us localize the regions of frequent successful one-pixel attacks at the image space.