GradDiv: Adversarial Robustness of Randomized Neural Networks via Gradient Diversity Regularization
This addresses the problem of adversarial robustness in deep learning for practitioners using randomized neural networks, offering an incremental improvement over existing defenses.
The paper tackles the vulnerability of randomized neural networks to adversarial attacks using proxy gradients by showing that attacks are less effective when gradients are more scattered, and proposes Gradient Diversity (GradDiv) regularizations to minimize gradient concentration, improving robustness on MNIST, CIFAR10, and STL10 datasets against state-of-the-art attacks.
Deep learning is vulnerable to adversarial examples. Many defenses based on randomized neural networks have been proposed to solve the problem, but fail to achieve robustness against attacks using proxy gradients such as the Expectation over Transformation (EOT) attack. We investigate the effect of the adversarial attacks using proxy gradients on randomized neural networks and demonstrate that it highly relies on the directional distribution of the loss gradients of the randomized neural network. We show in particular that proxy gradients are less effective when the gradients are more scattered. To this end, we propose Gradient Diversity (GradDiv) regularizations that minimize the concentration of the gradients to build a robust randomized neural network. Our experiments on MNIST, CIFAR10, and STL10 show that our proposed GradDiv regularizations improve the adversarial robustness of randomized neural networks against a variety of state-of-the-art attack methods. Moreover, our method efficiently reduces the transferability among sample models of randomized neural networks.