A Priori Estimates of the Population Risk for Two-layer Neural Networks
This work addresses the theoretical understanding of neural network performance for researchers in machine learning, though it appears incremental as it builds on existing risk estimation frameworks.
The authors derived new a priori estimates for the population risk of two-layer neural networks, achieving nearly optimal error rates that scale like Monte Carlo rates and remain effective in over-parametrized regimes. They used these estimates to explain why neural networks outperform related kernel methods.
New estimates for the population risk are established for two-layer neural networks. These estimates are nearly optimal in the sense that the error rates scale in the same way as the Monte Carlo error rates. They are equally effective in the over-parametrized regime when the network size is much larger than the size of the dataset. These new estimates are a priori in nature in the sense that the bounds depend only on some norms of the underlying functions to be fitted, not the parameters in the model, in contrast with most existing results which are a posteriori in nature. Using these a priori estimates, we provide a perspective for understanding why two-layer neural networks perform better than the related kernel methods.