Generalization bounds for deep convolutional neural networks
This work addresses the theoretical challenge of generalization bounds for deep learning practitioners, but it is incremental as it builds on existing frameworks.
The paper tackles the problem of understanding generalization in deep convolutional neural networks by proving bounds on generalization error in terms of training loss, parameters, Lipschitz constant, and weight distances, independent of input size and feature map dimensions, with experiments on CIFAR-10 showing comparisons to practical gaps.
We prove bounds on the generalization error of convolutional networks. The bounds are in terms of the training loss, the number of parameters, the Lipschitz constant of the loss and the distance from the weights to the initial weights. They are independent of the number of pixels in the input, and the height and width of hidden feature maps. We present experiments using CIFAR-10 with varying hyperparameters of a deep convolutional network, comparing our bounds with practical generalization gaps.