Architecture independent generalization bounds for overparametrized deep ReLU networks
This provides theoretical guarantees for generalization in overparametrized deep learning, addressing a fundamental problem in machine learning theory.
The paper proves that overparametrized deep ReLU networks can generalize with test error independent of overparametrization and VC dimension, showing explicit bounds that depend only on data geometry, activation regularity, and weight/bias norms. For networks with training sample size bounded by input dimension, it constructs zero-loss minimizers without gradient descent and proves architecture-independent generalization error.
We prove that overparametrized neural networks are able to generalize with a test error that is independent of the level of overparametrization, and independent of the Vapnik-Chervonenkis (VC) dimension. We prove explicit bounds that only depend on the metric geometry of the test and training sets, on the regularity properties of the activation function, and on the operator norms of the weights and norms of biases. For overparametrized deep ReLU networks with a training sample size bounded by the input space dimension, we explicitly construct zero loss minimizers without use of gradient descent, and prove that the generalization error is independent of the network architecture.