Schauder Bases for $C[0, 1]$ Using ReLU, Softplus and Two Sigmoidal Functions
This provides a foundational mathematical framework for neural network approximation theory, though it is incremental in extending known basis constructions to specific activation functions.
The paper tackled the problem of constructing Schauder bases for the space C[0,1] using ReLU, Softplus, and sigmoidal functions, establishing their existence for the first time and improving on universal approximation properties.
We construct four Schauder bases for the space $C[0,1]$, one using ReLU functions, another using Softplus functions, and two more using sigmoidal versions of the ReLU and Softplus functions. This establishes the existence of a basis using these functions for the first time, and improves on the universal approximation property associated with them.