Neural network approximation and estimation of classifiers with classification boundary in a Barron class
This work provides theoretical performance guarantees for neural network classifiers, which is significant for researchers and practitioners developing and deploying machine learning models, especially in high-dimensional settings.
This paper establishes approximation and estimation bounds for binary classifiers using ReLU neural networks, demonstrating that the curse of dimensionality can be overcome as rates are largely independent of input dimension. The target classification function's interfaces are assumed to be locally of Barron-type, and the study also clarifies relationships between different Barron-type spaces.
We prove bounds for the approximation and estimation of certain binary classification functions using ReLU neural networks. Our estimation bounds provide a priori performance guarantees for empirical risk minimization using networks of a suitable size, depending on the number of training samples available. The obtained approximation and estimation rates are independent of the dimension of the input, showing that the curse of dimensionality can be overcome in this setting; in fact, the input dimension only enters in the form of a polynomial factor. Regarding the regularity of the target classification function, we assume the interfaces between the different classes to be locally of Barron-type. We complement our results by studying the relations between various Barron-type spaces that have been proposed in the literature. These spaces differ substantially more from each other than the current literature suggests.