Efficient uniform approximation using Random Vector Functional Link networks
This addresses the problem of efficient function approximation for researchers in machine learning theory, offering a theoretical guarantee for RVFL networks with specific random weights, though it is incremental as it builds on existing RVFL frameworks.
The paper proves that Random Vector Functional Link (RVFL) networks with ReLU activations can approximate Lipschitz continuous functions in the L∞ norm, providing a nonasymptotic lower bound on the number of hidden nodes needed to achieve a given accuracy with high probability.
A Random Vector Functional Link (RVFL) network is a depth-2 neural network with random inner weights and biases. Only the outer weights of such an architecture are to be learned, so the learning process boils down to a linear optimization task, allowing one to sidestep the pitfalls of nonconvex optimization problems. In this paper, we prove that an RVFL with ReLU activation functions can approximate Lipschitz continuous functions in $L_\infty$ norm. To the best of our knowledge, our result is the first approximation result in $L_\infty$ norm using nice inner weights; namely, Gaussians. We give a nonasymptotic lower bound for the number of hidden-layer nodes to achieve a given accuracy with high probability, depending on, among other things, the Lipschitz constant of the target function, the desired accuracy, and the input dimension. Our method of proof is rooted in probability theory and harmonic analysis.