Adversarial Robustness via Label-Smoothing
This work addresses adversarial robustness for deep-learning practitioners, offering an incremental improvement through variations of an existing technique.
The paper tackles the problem of improving adversarial robustness in supervised deep-learning models by proposing Label-Smoothing methods, showing that these methods enhance robustness against various attacks on multiple datasets and models without increasing training time or modifying network architecture.
We study Label-Smoothing as a means for improving adversarial robustness of supervised deep-learning models. After establishing a thorough and unified framework, we propose several variations to this general method: adversarial, Boltzmann and second-best Label-Smoothing methods, and we explain how to construct your own one. On various datasets (MNIST, CIFAR10, SVHN) and models (linear models, MLPs, LeNet, ResNet), we show that Label-Smoothing in general improves adversarial robustness against a variety of attacks (FGSM, BIM, DeepFool, Carlini-Wagner) by better taking account of the dataset geometry. The proposed Label-Smoothing methods have two main advantages: they can be implemented as a modified cross-entropy loss, thus do not require any modifications of the network architecture nor do they lead to increased training times, and they improve both standard and adversarial accuracy.