Random Projection Neural Networks of Best Approximation: Convergence theory and practical applications
This work addresses efficient and accurate function approximation for computational applications, but it appears incremental as it builds on existing random projection neural network methods.
The paper tackles the problem of function approximation using feedforward neural networks with random projections, demonstrating that with non-polynomial activation functions, these networks can achieve exponential convergence rates for infinitely differentiable functions, and tests show comparable performance to Legendre Polynomials on five benchmark problems.
We investigate the concept of Best Approximation for Feedforward Neural Networks (FNN) and explore their convergence properties through the lens of Random Projection (RPNNs). RPNNs have predetermined and fixed, once and for all, internal weights and biases, offering computational efficiency. We demonstrate that there exists a choice of external weights, for any family of such RPNNs, with non-polynomial infinitely differentiable activation functions, that exhibit an exponential convergence rate when approximating any infinitely differentiable function. For illustration purposes, we test the proposed RPNN-based function approximation, with parsimoniously chosen basis functions, across five benchmark function approximation problems. Results show that RPNNs achieve comparable performance to established methods such as Legendre Polynomials, highlighting their potential for efficient and accurate function approximation.