Efficient and Effective Augmentation Strategy for Adversarial Training
This addresses the problem of improving adversarial robustness in deep neural networks for image classification, offering a novel approach to enhance data efficiency and performance, though it appears incremental as it builds on existing adversarial training methods.
The paper tackles the challenge of effectively using data augmentations in adversarial training, which is more data-hungry than standard training, by proposing DAJAT, a method that combines simple and complex augmentations with separate batch normalization and a Jensen-Shannon divergence loss. The result is a substantially better robustness-accuracy trade-off on the RobustBench Leaderboard for ResNet-18 and WideResNet-34-10.
Adversarial training of Deep Neural Networks is known to be significantly more data-hungry when compared to standard training. Furthermore, complex data augmentations such as AutoAugment, which have led to substantial gains in standard training of image classifiers, have not been successful with Adversarial Training. We first explain this contrasting behavior by viewing augmentation during training as a problem of domain generalization, and further propose Diverse Augmentation-based Joint Adversarial Training (DAJAT) to use data augmentations effectively in adversarial training. We aim to handle the conflicting goals of enhancing the diversity of the training dataset and training with data that is close to the test distribution by using a combination of simple and complex augmentations with separate batch normalization layers during training. We further utilize the popular Jensen-Shannon divergence loss to encourage the joint learning of the diverse augmentations, thereby allowing simple augmentations to guide the learning of complex ones. Lastly, to improve the computational efficiency of the proposed method, we propose and utilize a two-step defense, Ascending Constraint Adversarial Training (ACAT), that uses an increasing epsilon schedule and weight-space smoothing to prevent gradient masking. The proposed method DAJAT achieves substantially better robustness-accuracy trade-off when compared to existing methods on the RobustBench Leaderboard on ResNet-18 and WideResNet-34-10. The code for implementing DAJAT is available here: https://github.com/val-iisc/DAJAT.