Adversarial VC-dimension and Sample Complexity of Neural Networks
This work addresses the challenge of adversarial robustness for neural networks in security-critical applications, providing a theoretical foundation, though it appears incremental as it builds on existing VC-dimension concepts.
The paper tackled the problem of understanding the theoretical limits of neural networks under adversarial attacks by deriving the adversarial VC-dimension for neural networks with sign activation functions, establishing a relationship between the growth number of the network and its neurons.
Adversarial attacks during the testing phase of neural networks pose a challenge for the deployment of neural networks in security critical settings. These attacks can be performed by adding noise that is imperceptible to humans on top of the original data. By doing so, an attacker can create an adversarial sample, which will cause neural networks to misclassify. In this paper, we seek to understand the theoretical limits of what can be learned by neural networks in the presence of an adversary. We first defined the hypothesis space of a neural network, and showed the relationship between the growth number of the entire neural network and the growth number of each neuron. Combine that with the adversarial Vapnik-Chervonenkis(VC)-dimension of halfspace classifiers, we concluded the adversarial VC-dimension of the neural networks with sign activation functions.