Spectral methods for Neural Integral Equations
This work addresses computational bottlenecks for researchers using neural integral equations in machine learning, though it appears incremental as it builds on existing methods with a spectral approach.
The authors tackled the computational expense of neural integral equations by introducing a spectral methods framework that learns operators in the spectral domain, resulting in cheaper computational costs and high interpolation accuracy, with theoretical guarantees and numerical experiments demonstrating practical effectiveness.
Neural integral equations are deep learning models based on the theory of integral equations, where the model consists of an integral operator and the corresponding equation (of the second kind) which is learned through an optimization procedure. This approach allows to leverage the nonlocal properties of integral operators in machine learning, but it is computationally expensive. In this article, we introduce a framework for neural integral equations based on spectral methods that allows us to learn an operator in the spectral domain, resulting in a cheaper computational cost, as well as in high interpolation accuracy. We study the properties of our methods and show various theoretical guarantees regarding the approximation capabilities of the model, and convergence to solutions of the numerical methods. We provide numerical experiments to demonstrate the practical effectiveness of the resulting model.