Masking Adversarial Damage: Finding Adversarial Saliency for Robust and Sparse Network
This work addresses the challenge of combining adversarial robustness with model compression to reduce computational and memory costs for practitioners, though it is incremental as it builds on existing adversarial training and pruning techniques.
The paper tackles the problem of adversarial robustness in deep neural networks by proposing a novel adversarial pruning method called Masking Adversarial Damage (MAD), which uses second-order information to estimate adversarial saliency and prune parameters without compromising robustness, achieving better performance than previous methods on three public datasets.
Adversarial examples provoke weak reliability and potential security issues in deep neural networks. Although adversarial training has been widely studied to improve adversarial robustness, it works in an over-parameterized regime and requires high computations and large memory budgets. To bridge adversarial robustness and model compression, we propose a novel adversarial pruning method, Masking Adversarial Damage (MAD) that employs second-order information of adversarial loss. By using it, we can accurately estimate adversarial saliency for model parameters and determine which parameters can be pruned without weakening adversarial robustness. Furthermore, we reveal that model parameters of initial layer are highly sensitive to the adversarial examples and show that compressed feature representation retains semantic information for the target objects. Through extensive experiments on three public datasets, we demonstrate that MAD effectively prunes adversarially trained networks without loosing adversarial robustness and shows better performance than previous adversarial pruning methods.