Toward a Better Understanding of Fourier Neural Operators from a Spectral Perspective
This addresses a limitation in PDE modeling for scientific computing, but it is incremental as it builds on existing FNO methods.
The paper tackled the problem of Fourier Neural Operators (FNOs) being ineffective with large Fourier kernels in solving PDEs, and proposed SpecB-FNO to mitigate a Fourier parameterization bias, achieving an average 50% improvement in prediction accuracy.
In solving partial differential equations (PDEs), Fourier Neural Operators (FNOs) have exhibited notable effectiveness. However, FNO is observed to be ineffective with large Fourier kernels that parameterize more frequencies. Current solutions rely on setting small kernels, restricting FNO's ability to capture complex PDE data in real-world applications. This paper offers empirical insights into FNO's difficulty with large kernels through spectral analysis: FNO exhibits a unique Fourier parameterization bias, excelling at learning dominant frequencies in target data while struggling with non-dominant frequencies. To mitigate such a bias, we propose SpecB-FNO to enhance the capture of non-dominant frequencies by adopting additional residual modules to learn from the previous ones' prediction residuals iteratively. By effectively utilizing large Fourier kernels, SpecB-FNO achieves better prediction accuracy on diverse PDE applications, with an average improvement of 50%.