Some Super-approximation Rates of ReLU Neural Networks for Korobov Functions
This work addresses the approximation theory problem for high-dimensional functions, providing incremental improvements in error bounds for researchers in numerical analysis and machine learning.
This paper tackles the approximation of Korobov functions using ReLU neural networks, deriving nearly optimal super-approximation error bounds of order 2m in L_p norm and 2m-2 in W^1_p norm, which improve upon classical bounds and show that neural network expressivity avoids the curse of dimensionality.
This paper examines the $L_p$ and $W^1_p$ norm approximation errors of ReLU neural networks for Korobov functions. In terms of network width and depth, we derive nearly optimal super-approximation error bounds of order $2m$ in the $L_p$ norm and order $2m-2$ in the $W^1_p$ norm, for target functions with $L_p$ mixed derivative of order $m$ in each direction. The analysis leverages sparse grid finite elements and the bit extraction technique. Our results improve upon classical lowest order $L_\infty$ and $H^1$ norm error bounds and demonstrate that the expressivity of neural networks is largely unaffected by the curse of dimensionality.