A Survey on Universal Approximation Theorems
It synthesizes existing knowledge on UATs, which is foundational for understanding neural network capabilities, but is incremental as a survey.
This paper provides a systematic overview of universal approximation theorems (UATs) for neural networks, covering theoretical and numerical aspects from both arbitrary width and depth perspectives.
This paper discusses various theorems on the approximation capabilities of neural networks (NNs), which are known as universal approximation theorems (UATs). The paper gives a systematic overview of UATs starting from the preliminary results on function approximation, such as Taylor's theorem, Fourier's theorem, Weierstrass approximation theorem, Kolmogorov - Arnold representation theorem, etc. Theoretical and numerical aspects of UATs are covered from both arbitrary width and depth.