An elementary proof of a universal approximation theorem
This is an incremental contribution that simplifies the proof of a known theorem for researchers and students in machine learning theory.
The authors tackled the problem of proving a universal approximation theorem for neural networks with three hidden layers and increasing, continuous, bounded activation functions, providing an elementary proof that requires only undergraduate analysis, though the result is weaker than the best known ones.
In this short note, we give an elementary proof of a universal approximation theorem for neural networks with three hidden layers and increasing, continuous, bounded activation function. The result is weaker than the best known results, but the proof is elementary in the sense that no machinery beyond undergraduate analysis is used.