Orthogonal greedy algorithm for linear operator learning with shallow neural network
This work addresses kernel estimation for linear PDEs and operator learning, offering incremental improvements over existing methods.
The paper tackles the problem of learning linear operators, equivalent to kernel estimation, by extending the orthogonal greedy algorithm (OGA) to this domain, achieving orders of accuracy improvement in tasks such as approximating Green's functions for PDEs.
Greedy algorithms, particularly the orthogonal greedy algorithm (OGA), have proven effective in training shallow neural networks for fitting functions and solving partial differential equations (PDEs). In this paper, we extend the application of OGA to the tasks of linear operator learning, which is equivalent to learning the kernel function through integral transforms. Firstly, a novel greedy algorithm is developed for kernel estimation rate in a new semi-inner product, which can be utilized to approximate the Green's function of linear PDEs from data. Secondly, we introduce the OGA for point-wise kernel estimation to further improve the approximation rate, achieving orders of accuracy improvement across various tasks and baseline models. In addition, we provide a theoretical analysis on the kernel estimation problem and the optimal approximation rates for both algorithms, establishing their efficacy and potential for future applications in PDEs and operator learning tasks.