Optimal learning of high-dimensional classification problems using deep neural networks
This provides theoretical guarantees for deep learning in high-dimensional classification, though it is incremental as it builds on existing regularity assumptions.
The paper establishes universal lower bounds for learning classification functions from noiseless data with regular decision boundaries, showing that for Barron-regular boundaries, optimal estimation rates are dimension-independent and achievable via deep neural networks.
We study the problem of learning classification functions from noiseless training samples, under the assumption that the decision boundary is of a certain regularity. We establish universal lower bounds for this estimation problem, for general classes of continuous decision boundaries. For the class of locally Barron-regular decision boundaries, we find that the optimal estimation rates are essentially independent of the underlying dimension and can be realized by empirical risk minimization methods over a suitable class of deep neural networks. These results are based on novel estimates of the $L^1$ and $L^\infty$ entropies of the class of Barron-regular functions.