Characterization of the Variation Spaces Corresponding to Shallow Neural Networks
This work provides theoretical insights into approximation theory for shallow neural networks, which is incremental as it clarifies existing mathematical frameworks.
The paper tackles the problem of characterizing variation spaces for shallow neural networks by comparing definitions based on convex hulls and integral representations, showing that three key spaces (Barron, spectral Barron, and Radon BV) are variation spaces with respect to specific dictionaries.
We study the variation space corresponding to a dictionary of functions in $L^2(Ω)$ for a bounded domain $Ω\subset \mathbb{R}^d$. Specifically, we compare the variation space, which is defined in terms of a convex hull with related notions based on integral representations. This allows us to show that three important notions relating to the approximation theory of shallow neural networks, the Barron space, the spectral Barron space, and the Radon BV space, are actually variation spaces with respect to certain natural dictionaries.