An Orthogonal Classifier for Improving the Adversarial Robustness of Neural Networks
This work addresses the adversarial robustness problem for neural network users, offering an incremental improvement by modifying the classification layer to enhance defense against attacks.
The paper tackles the problem of neural networks' vulnerability to adversarial perturbations by proposing a novel robust classifier with a dense orthogonal weight matrix of uniform magnitude, which improves robustness without structural redundancy. Experimental results show the method is efficient and competitive with state-of-the-art defenses, achieving high accuracy on clean data and better robustness when adversarial samples are used.
Neural networks are susceptible to artificially designed adversarial perturbations. Recent efforts have shown that imposing certain modifications on classification layer can improve the robustness of the neural networks. In this paper, we explicitly construct a dense orthogonal weight matrix whose entries have the same magnitude, thereby leading to a novel robust classifier. The proposed classifier avoids the undesired structural redundancy issue in previous work. Applying this classifier in standard training on clean data is sufficient to ensure the high accuracy and good robustness of the model. Moreover, when extra adversarial samples are used, better robustness can be further obtained with the help of a special worst-case loss. Experimental results show that our method is efficient and competitive to many state-of-the-art defensive approaches. Our code is available at \url{https://github.com/MTandHJ/roboc}.