The Boundaries of Verifiable Accuracy, Robustness, and Generalisation in Deep Learning
This work addresses fundamental verification challenges in deep learning, which is crucial for ensuring reliability in safety-critical applications, but it is incremental as it builds on existing theoretical frameworks.
The paper investigates the theoretical limitations of verifying guaranteed stability, accuracy, and generalization in neural networks for classification tasks, showing that computing and verifying ideal solutions is extremely challenging for a large family of tasks, even when such solutions exist within the given architectures.
In this work, we assess the theoretical limitations of determining guaranteed stability and accuracy of neural networks in classification tasks. We consider classical distribution-agnostic framework and algorithms minimising empirical risks and potentially subjected to some weights regularisation. We show that there is a large family of tasks for which computing and verifying ideal stable and accurate neural networks in the above settings is extremely challenging, if at all possible, even when such ideal solutions exist within the given class of neural architectures.