A single hidden layer feedforward network with only one neuron in the hidden layer can approximate any univariate function
This provides a theoretical foundation for simplifying neural network architectures in function approximation, though it is incremental as it builds on existing work on universal approximation.
The paper tackles the problem of approximating arbitrary continuous univariate functions using a single hidden layer feedforward network with only one neuron, and constructs a smooth, sigmoidal activation function that achieves this approximation to any desired accuracy, with implementation in a computer program.
The possibility of approximating a continuous function on a compact subset of the real line by a feedforward single hidden layer neural network with a sigmoidal activation function has been studied in many papers. Such networks can approximate an arbitrary continuous function provided that an unlimited number of neurons in a hidden layer is permitted. In this paper, we consider constructive approximation on any finite interval of $\mathbb{R}$ by neural networks with only one neuron in the hidden layer. We construct algorithmically a smooth, sigmoidal, almost monotone activation function $σ$ providing approximation to an arbitrary continuous function within any degree of accuracy. This algorithm is implemented in a computer program, which computes the value of $σ$ at any reasonable point of the real axis.