A Generalization Bound for Nearly-Linear Networks
This provides theoretical guarantees for generalization in neural networks, particularly for those close to linear, which is incremental as it builds on prior work but introduces a novel a-priori property.
The paper tackles the problem of deriving non-vacuous generalization bounds for neural networks by treating them as perturbations of linear networks, resulting in bounds that are a-priori and do not require training for evaluation.
We consider nonlinear networks as perturbations of linear ones. Based on this approach, we present novel generalization bounds that become non-vacuous for networks that are close to being linear. The main advantage over the previous works which propose non-vacuous generalization bounds is that our bounds are a-priori: performing the actual training is not required for evaluating the bounds. To the best of our knowledge, they are the first non-vacuous generalization bounds for neural nets possessing this property.