Incremental Spatial and Spectral Learning of Neural Operators for Solving Large-Scale PDEs
This incremental method improves efficiency and accuracy for solving large-scale PDEs like fluid flows, benefiting researchers in computational physics and engineering.
The paper tackles the computational and overfitting challenges in training Fourier Neural Operators (FNO) for large-scale PDEs by introducing an incremental learning approach (iFNO) that progressively increases frequency modes and data resolution, resulting in a 10% lower testing error, 20% fewer frequency modes, and 30% faster training compared to FNO.
Fourier Neural Operators (FNO) offer a principled approach to solving challenging partial differential equations (PDE) such as turbulent flows. At the core of FNO is a spectral layer that leverages a discretization-convergent representation in the Fourier domain, and learns weights over a fixed set of frequencies. However, training FNO presents two significant challenges, particularly in large-scale, high-resolution applications: (i) Computing Fourier transform on high-resolution inputs is computationally intensive but necessary since fine-scale details are needed for solving many PDEs, such as fluid flows, (ii) selecting the relevant set of frequencies in the spectral layers is challenging, and too many modes can lead to overfitting, while too few can lead to underfitting. To address these issues, we introduce the Incremental Fourier Neural Operator (iFNO), which progressively increases both the number of frequency modes used by the model as well as the resolution of the training data. We empirically show that iFNO reduces total training time while maintaining or improving generalization performance across various datasets. Our method demonstrates a 10% lower testing error, using 20% fewer frequency modes compared to the existing Fourier Neural Operator, while also achieving a 30% faster training.