Approximation of RKHS Functionals by Neural Networks
This addresses the challenge of integrating functional data like time series and images into neural networks for learning maps from function spaces to real numbers, offering a simpler approach compared to existing methods that rely on integration-type basis expansions.
The paper tackles the problem of approximating functionals on reproducing kernel Hilbert spaces (RKHS's) using neural networks, establishing universality and deriving explicit error bounds for specific kernels, and applies this to functional regression to show accurate approximation of regression maps in generalized functional linear models.
Motivated by the abundance of functional data such as time series and images, there has been a growing interest in integrating such data into neural networks and learning maps from function spaces to R (i.e., functionals). In this paper, we study the approximation of functionals on reproducing kernel Hilbert spaces (RKHS's) using neural networks. We establish the universality of the approximation of functionals on the RKHS's. Specifically, we derive explicit error bounds for those induced by inverse multiquadric, Gaussian, and Sobolev kernels. Moreover, we apply our findings to functional regression, proving that neural networks can accurately approximate the regression maps in generalized functional linear models. Existing works on functional learning require integration-type basis function expansions with a set of pre-specified basis functions. By leveraging the interpolating orthogonal projections in RKHS's, our proposed network is much simpler in that we use point evaluations to replace basis function expansions.