Effects of Loss Functions And Target Representations on Adversarial Robustness
This work addresses the vulnerability of neural networks to adversarial attacks, which is a critical security issue for AI systems, though it is incremental as it builds on existing attack methods.
The paper tackles the problem of adversarial robustness in neural networks by exploring alternative loss functions and target representations, specifically mean-squared error and codewords from a random codebook, resulting in up to 98.7% higher accuracy against untargeted attacks and up to 99.8% lower success rates for targeted attacks.
Understanding and evaluating the robustness of neural networks under adversarial settings is a subject of growing interest. Attacks proposed in the literature usually work with models trained to minimize cross-entropy loss and output softmax probabilities. In this work, we present interesting experimental results that suggest the importance of considering other loss functions and target representations, specifically, (1) training on mean-squared error and (2) representing targets as codewords generated from a random codebook. We evaluate the robustness of neural networks that implement these proposed modifications using existing attacks, showing an increase in accuracy against untargeted attacks of up to 98.7\% and a decrease of targeted attack success rates of up to 99.8\%. Our model demonstrates more robustness compared to its conventional counterpart even against attacks that are tailored to our modifications. Furthermore, we find that the parameters of our modified model have significantly smaller Lipschitz bounds, an important measure correlated with a model's sensitivity to adversarial perturbations.