Higher Order Approximation Rates for ReLU CNNs in Korobov Spaces
This addresses the expressivity of CNNs in high-dimensional function approximation, showing reduced dimensionality curse, but is incremental as it builds on known methods for specific function classes.
The paper tackles the approximation error of ReLU CNNs for higher order Korobov functions, improving the classical second-order rate to an (m+1)-th order rate (with a logarithmic factor) in terms of network depth.
This paper investigates the $L_p$ approximation error for higher order Korobov functions using deep convolutional neural networks (CNNs) with ReLU activation. For target functions having a mixed derivative of order m+1 in each direction, we improve classical approximation rate of second order to (m+1)-th order (modulo a logarithmic factor) in terms of the depth of CNNs. The key ingredient in our analysis is approximate representation of high-order sparse grid basis functions by CNNs. The results suggest that higher order expressivity of CNNs does not severely suffer from the curse of dimensionality.