Frequency Regularization for Improving Adversarial Robustness
This addresses the problem of improving adversarial robustness for deep learning models, particularly in computer vision, with an incremental improvement over existing adversarial training methods.
The paper tackles the vulnerability of deep neural networks to adversarial attacks by proposing frequency regularization (FR) to align output differences in the spectral domain, combined with Stochastic Weight Averaging (SWA), achieving the strongest robustness against attacks like PGD-20, C&W, and Autoattack on CIFAR-10 without extra data.
Deep neural networks are incredibly vulnerable to crafted, human-imperceptible adversarial perturbations. Although adversarial training (AT) has proven to be an effective defense approach, we find that the AT-trained models heavily rely on the input low-frequency content for judgment, accounting for the low standard accuracy. To close the large gap between the standard and robust accuracies during AT, we investigate the frequency difference between clean and adversarial inputs, and propose a frequency regularization (FR) to align the output difference in the spectral domain. Besides, we find Stochastic Weight Averaging (SWA), by smoothing the kernels over epochs, further improves the robustness. Among various defense schemes, our method achieves the strongest robustness against attacks by PGD-20, C\&W and Autoattack, on a WideResNet trained on CIFAR-10 without any extra data.