Sharp Sobolev Sandwich and Approximation Rates of Radon-Domain $L^p$ Ridge Integral Spaces for ReLU$^k$ Networks
For researchers in approximation theory and neural networks, this work provides a rigorous functional analytic framework and optimal approximation rates for ReLU$^k$ networks, clarifying the role of activation regularity.
The paper establishes the $L^p$ space theory for shallow ReLU$^k$ networks, proving that the Radon-domain $L^p$ space $\mathcal{R}L^p_k(\Omega)$ is sandwiched between Sobolev spaces with sharp gaps determined by the Seeger–Sogge–Stein loss. As an application, they derive high-probability $L^p$ approximation rates and the optimal Hilbert rate $O(n^{-1/2 - (2k+1)/(2d)})$ at $p=2$ for linearized neural networks.
We develop the $L^p$ space and approximation theory for shallow neural networks with $\mathrm{ReLU}^k$ activations. The central object is the Radon-domain $L^p$ space $\mathcal{R}L^p_k(Ω)$ containing all functions on a bounded domain $Ω$ that admit a ridge integral representation whose coefficient density belongs to $L^p$ in the Radon domain. In the Hilbert case $p=2$, we prove by elementary Fourier analysis that this space recovers the critical Sobolev space $H^{k+(d+1)/2}(Ω)$. For general $1<p<\infty$, the identity becomes a Sobolev sandwich. The sharp gap of each side is exactly the Seeger--Sogge--Stein loss for the Radon transform as a Fourier integral operator. This also clarifies how the activation regularity and Radon back-projection jointly produce the regularity. As an application, we discretize the integral representation using a deterministic interpolation skeleton plus uniform sampling. This yields high-probability $L^p$ approximation rates and the optimal Hilbert rate $O\!\big(n^{-\frac12-\frac{2k+1}{2d}}\big)$ at $p=2$ for linearized neural networks.