Optimal Convergence Rates of Deep Convolutional Neural Networks: Additive Ridge Functions
This work demonstrates the statistical optimality of convolutional neural networks for high-dimensional regression, which is an incremental theoretical advancement for machine learning researchers.
The paper tackles the regression problem for deep convolutional neural networks by analyzing mean squared error, showing that for additive ridge functions, these networks with a fully connected layer achieve optimal mini-max convergence rates up to a log factor, with input dimension only affecting the constant.
Convolutional neural networks have shown impressive abilities in many applications, especially those related to the classification tasks. However, for the regression problem, the abilities of convolutional structures have not been fully understood, and further investigation is needed. In this paper, we consider the mean squared error analysis for deep convolutional neural networks. We show that, for additive ridge functions, convolutional neural networks followed by one fully connected layer with ReLU activation functions can reach optimal mini-max rates (up to a log factor). The input dimension only appears in the constant of convergence rates. This work shows the statistical optimality of convolutional neural networks and may shed light on why convolutional neural networks are able to behave well for high dimensional input.