Random Features for Operator-Valued Kernels: Bridging Kernel Methods and Neural Operators
This work provides a unified theoretical framework for analyzing neural operators and networks, bridging kernel methods and neural operators, but it is incremental as it builds on prior results.
The paper tackles the generalization properties of random feature methods by extending analysis to spectral regularization and operator-valued kernels, establishing optimal learning rates and neuron requirements for accuracy, and providing minimax rates for both well-specified and misspecified cases.
In this work, we investigate the generalization properties of random feature methods. Our analysis extends prior results for Tikhonov regularization to a broad class of spectral regularization techniques and further generalizes the setting to operator-valued kernels. This unified framework enables a rigorous theoretical analysis of neural operators and neural networks through the lens of the Neural Tangent Kernel (NTK). In particular, it allows us to establish optimal learning rates and provides a good understanding of how many neurons are required to achieve a given accuracy. Furthermore, we establish minimax rates in the well-specified case and also in the misspecified case, where the target is not contained in the reproducing kernel Hilbert space. These results sharpen and complete earlier findings for specific kernel algorithms.