A Frequency Perspective of Adversarial Robustness
This provides insights for researchers in adversarial machine learning to develop better attacks and defenses, though it is incremental in clarifying existing misconceptions.
The paper tackles the problem of understanding adversarial examples in deep learning by analyzing them from a frequency perspective, showing that they are dataset-dependent rather than high-frequency noise, with empirical evidence from CIFAR-10 and ImageNet datasets.
Adversarial examples pose a unique challenge for deep learning systems. Despite recent advances in both attacks and defenses, there is still a lack of clarity and consensus in the community about the true nature and underlying properties of adversarial examples. A deep understanding of these examples can provide new insights towards the development of more effective attacks and defenses. Driven by the common misconception that adversarial examples are high-frequency noise, we present a frequency-based understanding of adversarial examples, supported by theoretical and empirical findings. Our analysis shows that adversarial examples are neither in high-frequency nor in low-frequency components, but are simply dataset dependent. Particularly, we highlight the glaring disparities between models trained on CIFAR-10 and ImageNet-derived datasets. Utilizing this framework, we analyze many intriguing properties of training robust models with frequency constraints, and propose a frequency-based explanation for the commonly observed accuracy vs. robustness trade-off.