Negative results for approximation using single layer and multilayer feedforward neural networks
This addresses theoretical limitations in neural network approximation for researchers, highlighting fundamental constraints rather than incremental improvements.
The paper proves negative results for approximating functions on compact subsets of ℝ^d using feedforward neural networks, showing existence of target functions that are arbitrarily difficult to approximate with single-layer networks and arbitrary continuous activation functions, and extends this to multilayer networks with specific activation types.
We prove a negative result for the approximation of functions defined on compact subsets of $\mathbb{R}^d$ (where $d \geq 2$) using feedforward neural networks with one hidden layer and arbitrary continuous activation function. In a nutshell, this result claims the existence of target functions that are as difficult to approximate using these neural networks as one may want. We also demonstrate an analogous result (for general $d \in \mathbb{N}$) for neural networks with an \emph{arbitrary} number of hidden layers, for activation functions that are either rational functions or continuous splines with finitely many pieces.