The Fourier Spectral Transformer Networks For Efficient and Generalizable Nonlinear PDEs Prediction
This work addresses the challenge of real-time prediction and control of complex dynamical systems, such as fluid flows, by offering a generalizable and efficient method, though it appears incremental as it combines existing spectral methods and Transformer architectures.
The authors tackled the problem of predicting nonlinear partial differential equations (PDEs) by proposing a Fourier Spectral Transformer network, which achieved highly accurate long-term predictions on 2D Navier-Stokes and 1D Burgers' equations, outperforming traditional numerical and machine learning methods.
In this work we propose a unified Fourier Spectral Transformer network that integrates the strengths of classical spectral methods and attention based neural architectures. By transforming the original PDEs into spectral ordinary differential equations, we use high precision numerical solvers to generate training data and use a Transformer network to model the evolution of the spectral coefficients. We demonstrate the effectiveness of our approach on the two dimensional incompressible Navier-Stokes equations and the one dimensional Burgers' equation. The results show that our spectral Transformer can achieve highly accurate long term predictions even with limited training data, better than traditional numerical methods and machine learning methods in forecasting future flow dynamics. The proposed framework generalizes well to unseen data, bringing a promising paradigm for real time prediction and control of complex dynamical systems.