Dimension-independent learning rates for high-dimensional classification problems
This addresses high-dimensional classification problems, offering theoretical guarantees for neural network performance, but it appears incremental as it modifies existing results.
The paper tackles the problem of approximating and estimating classification functions with decision boundaries in the RBV^2 space, showing that neural networks can approximate these functions without the curse of dimensionality and providing dimension-independent learning rates.
We study the problem of approximating and estimating classification functions that have their decision boundary in the $RBV^2$ space. Functions of $RBV^2$ type arise naturally as solutions of regularized neural network learning problems and neural networks can approximate these functions without the curse of dimensionality. We modify existing results to show that every $RBV^2$ function can be approximated by a neural network with bounded weights. Thereafter, we prove the existence of a neural network with bounded weights approximating a classification function. And we leverage these bounds to quantify the estimation rates. Finally, we present a numerical study that analyzes the effect of different regularity conditions on the decision boundaries.