Higher-Order Fourier Neural Operator: Explicit Mode Mixer for Nonlinear PDEs
For researchers using neural operators for PDEs, HO-FNO provides a principled way to handle nonlinearities, improving accuracy and efficiency over existing methods.
The authors introduce the Higher-Order Fourier Neural Operator (HO-FNO), which extends FNO by adding explicit n-linear mode mixing to capture nonlinear PDE dynamics. HO-FNO consistently outperforms other spectral neural operators and matches or exceeds state-of-the-art transformers and state-space models, with a single layer outperforming up to 16-layer FNOs on highly nonlinear problems.
Neural operators provide deep neural networks for learning mappings between function spaces. Among them, the Fourier Neural Operator (FNO) is particularly effective: its spectral convolution relies on low-dimensional Fourier-domain representations and can handle inputs at different resolutions. This design aligns well with settings where the Fourier basis diagonalizes the underlying operator, such as linear, constant-coefficient PDEs on periodic domains, in which Fourier modes evolve independently. However, nonlinear PDEs may benefit from an additional inductive bias, as they exhibit structured interactions between modes, governed by polynomial nonlinearities. To capture this inductive bias, we introduce the Higher-Order Spectral Convolution, a spectral mixer that extends FNO from diagonal modulation to explicit n-linear mode mixing, aligned with the dynamics of nonlinear PDEs. Our experiments on standard benchmarks show that the proposed Higher-Order FNO (HO-FNO) retains the efficiency of FNO-based architectures and consistently improves over other spectral neural operators. HO-FNO also performs on par with or better than state-of-the-art transformers and state-space models on several datasets, with stronger gains in highly nonlinear regimes, such as the Poisson equation with polynomial forcing, where a single HO-FNO layer outperforms FNO models with up to 16 layers. We open-source our code for reproducibility at: https://github.com/AlexColagrande/HO-FNO.