Over-parameterization and Adversarial Robustness in Neural Networks: An Overview and Empirical Analysis
This addresses the problem of understanding adversarial robustness in neural networks for researchers and practitioners, but it is incremental as it builds on prior contradictory findings.
The paper tackles the contradictory claims about whether over-parameterized neural networks are vulnerable to adversarial examples by empirically analyzing their robustness while evaluating attack reliability. The results show that over-parameterized networks are robust against adversarial attacks compared to under-parameterized ones, though no concrete numbers are provided.
Thanks to their extensive capacity, over-parameterized neural networks exhibit superior predictive capabilities and generalization. However, having a large parameter space is considered one of the main suspects of the neural networks' vulnerability to adversarial example -- input samples crafted ad-hoc to induce a desired misclassification. Relevant literature has claimed contradictory remarks in support of and against the robustness of over-parameterized networks. These contradictory findings might be due to the failure of the attack employed to evaluate the networks' robustness. Previous research has demonstrated that depending on the considered model, the algorithm employed to generate adversarial examples may not function properly, leading to overestimating the model's robustness. In this work, we empirically study the robustness of over-parameterized networks against adversarial examples. However, unlike the previous works, we also evaluate the considered attack's reliability to support the results' veracity. Our results show that over-parameterized networks are robust against adversarial attacks as opposed to their under-parameterized counterparts.