Learning Ability of Interpolating Deep Convolutional Neural Networks
It provides theoretical verification for how overfitted DCNNs generalize well, addressing a foundational problem in machine learning theory.
This paper studies the learning ability of deep convolutional neural networks (DCNNs) in both underparameterized and overparameterized settings, establishing the first learning rates for underparameterized DCNNs without restrictions and showing that interpolating DCNNs can maintain good learning rates through a novel network deepening scheme.
It is frequently observed that overparameterized neural networks generalize well. Regarding such phenomena, existing theoretical work mainly devotes to linear settings or fully-connected neural networks. This paper studies the learning ability of an important family of deep neural networks, deep convolutional neural networks (DCNNs), under both underparameterized and overparameterized settings. We establish the first learning rates of underparameterized DCNNs without parameter or function variable structure restrictions presented in the literature. We also show that by adding well-defined layers to a non-interpolating DCNN, we can obtain some interpolating DCNNs that maintain the good learning rates of the non-interpolating DCNN. This result is achieved by a novel network deepening scheme designed for DCNNs. Our work provides theoretical verification of how overfitted DCNNs generalize well.