Universal approximation results for neural networks with non-polynomial activation function over non-compact domains
This work addresses the theoretical limitation of neural network approximation to compact domains, which is important for applications in unbounded domains like PDEs and signal processing, though it is an incremental extension of existing theory.
The paper tackles the problem of extending the universal approximation property of neural networks to non-compact domains, establishing that single-hidden-layer networks with non-polynomial activation functions can approximate functions in spaces like L^p, weighted C^k, and weighted Sobolev spaces over unbounded domains, and provides dimension-independent approximation rates for functions with regular Fourier transforms.
This paper extends the universal approximation property of single-hidden-layer feedforward neural networks beyond compact domains, which is of particular interest for the approximation within weighted $C^k$-spaces and weighted Sobolev spaces over unbounded domains. More precisely, by assuming that the activation function is non-polynomial, we establish universal approximation results within function spaces defined over non-compact subsets of a Euclidean space, including $L^p$-spaces, weighted $C^k$-spaces, and weighted Sobolev spaces, where the latter two include the approximation of the (weak) derivatives. Moreover, we provide some dimension-independent rates for approximating a function with sufficiently regular and integrable Fourier transform by neural networks with non-polynomial activation function.