Latent Neural Operator Pretraining for Solving Time-Dependent PDEs
This work addresses the challenge of limited data for solving PDEs in scientific computing, offering a method that improves accuracy and efficiency, though it is incremental as it builds on existing neural operator techniques.
The authors tackled the data scarcity problem in solving time-dependent PDEs with neural operators by proposing a pretraining framework that reduces solution error by 31.7% on four problems, improving to 57.1% after fine-tuning, and achieves roughly 50% lower error and 3x data efficiency on out-of-distribution datasets.
Pretraining methods gain increasing attraction recently for solving PDEs with neural operators. It alleviates the data scarcity problem encountered by neural operator learning when solving single PDE via training on large-scale datasets consisting of various PDEs and utilizing shared patterns among different PDEs to improve the solution precision. In this work, we propose the Latent Neural Operator Pretraining (LNOP) framework based on the Latent Neural Operator (LNO) backbone. We achieve universal transformation through pretraining on hybrid time-dependent PDE dataset to extract representations of different physical systems and solve various time-dependent PDEs in the latent space through finetuning on single PDE dataset. Our proposed LNOP framework reduces the solution error by 31.7% on four problems and can be further improved to 57.1% after finetuning. On out-of-distribution dataset, our LNOP model achieves roughly 50% lower error and 3$\times$ data efficiency on average across different dataset sizes. These results show that our method is more competitive in terms of solution precision, transfer capability and data efficiency compared to non-pretrained neural operators.