Generalization Error Analysis of Neural networks with Gradient Based Regularization
This work addresses the challenge of improving neural network generalization and robustness for image classification, but it is incremental as it builds on existing regularization techniques with a new analytical framework.
The authors tackled the problem of analyzing generalization error in neural networks using gradient-based regularization methods, specifically total variation and Tikhonov regularization, and found through experiments that these methods significantly improve generalization ability and adversarial robustness in image classification tasks.
We study gradient-based regularization methods for neural networks. We mainly focus on two regularization methods: the total variation and the Tikhonov regularization. Applying these methods is equivalent to using neural networks to solve some partial differential equations, mostly in high dimensions in practical applications. In this work, we introduce a general framework to analyze the generalization error of regularized networks. The error estimate relies on two assumptions on the approximation error and the quadrature error. Moreover, we conduct some experiments on the image classification tasks to show that gradient-based methods can significantly improve the generalization ability and adversarial robustness of neural networks. A graphical extension of the gradient-based methods are also considered in the experiments.