NPAT Null-Space Projected Adversarial Training Towards Zero Deterioration
This work addresses the problem of maintaining accuracy while defending against adversarial attacks for neural network users, though it is incremental as it builds on existing adversarial training methods.
The authors tackled the trade-off between robustness and accuracy in adversarial training by proposing NPAT algorithms that use null-space projection to constrain adversarial samples, achieving comparable robustness with almost no loss in generalization on CIFAR10 and SVHN datasets.
To mitigate the susceptibility of neural networks to adversarial attacks, adversarial training has emerged as a prevalent and effective defense strategy. Intrinsically, this countermeasure incurs a trade-off, as it sacrifices the model's accuracy in processing normal samples. To reconcile the trade-off, we pioneer the incorporation of null-space projection into adversarial training and propose two innovative Null-space Projection based Adversarial Training(NPAT) algorithms tackling sample generation and gradient optimization, named Null-space Projected Data Augmentation (NPDA) and Null-space Projected Gradient Descent (NPGD), to search for an overarching optimal solutions, which enhance robustness with almost zero deterioration in generalization performance. Adversarial samples and perturbations are constrained within the null-space of the decision boundary utilizing a closed-form null-space projector, effectively mitigating threat of attack stemming from unreliable features. Subsequently, we conducted experiments on the CIFAR10 and SVHN datasets and reveal that our methodology can seamlessly combine with adversarial training methods and obtain comparable robustness while keeping generalization close to a high-accuracy model.