Universal Consistency of Wide and Deep ReLU Neural Networks and Minimax Optimal Convergence Rates for Kolmogorov-Donoho Optimal Function Classes
This provides theoretical guarantees for neural network classifiers in broad statistical learning contexts, though it is incremental in extending existing frameworks.
The paper proves universal consistency for wide and deep ReLU neural network classifiers trained on logistic loss and establishes minimax optimal convergence rates for Kolmogorov-Donoho optimal function classes, applicable to general settings beyond smoothness assumptions.
In this paper, we prove the universal consistency of wide and deep ReLU neural network classifiers trained on the logistic loss. We also give sufficient conditions for a class of probability measures for which classifiers based on neural networks achieve minimax optimal rates of convergence. The result applies to a wide range of known function classes. In particular, while most previous works impose explicit smoothness assumptions on the regression function, our framework encompasses more general settings. The proposed neural networks are either the minimizers of the logistic loss or the $0$-$1$ loss. In the former case, they are interpolating classifiers that exhibit a benign overfitting behavior.