Estimation of a regression function on a manifold by fully connected deep neural networks
This provides a theoretical foundation for using deep learning in high-dimensional data with low intrinsic structure, though it is incremental as it builds on existing convergence analysis.
The paper tackles the problem of estimating a regression function when predictor data lies on a lower-dimensional manifold, showing that deep neural networks with ReLU activation achieve a convergence rate dependent on the manifold's intrinsic dimension rather than the ambient dimension.
Estimation of a regression function from independent and identically distributed data is considered. The $L_2$ error with integration with respect to the distribution of the predictor variable is used as the error criterion. The rate of convergence of least squares estimates based on fully connected spaces of deep neural networks with ReLU activation function is analyzed for smooth regression functions. It is shown that in case that the distribution of the predictor variable is concentrated on a manifold, these estimates achieve a rate of convergence which depends on the dimension of the manifold and not on the number of components of the predictor variable.