Towards Universal Neural Operators through Multiphysics Pretraining
This work addresses the problem of high training costs for neural operators in data-driven simulations, offering an incremental improvement through transfer learning.
The paper tackled the computational expense of training neural operators for physical simulations by investigating transformer-based models in a transfer learning setting, demonstrating their effectiveness in transferring knowledge across diverse PDE problems.
Although neural operators are widely used in data-driven physical simulations, their training remains computationally expensive. Recent advances address this issue via downstream learning, where a model pretrained on simpler problems is fine-tuned on more complex ones. In this research, we investigate transformer-based neural operators, which have previously been applied only to specific problems, in a more general transfer learning setting. We evaluate their performance across diverse PDE problems, including extrapolation to unseen parameters, incorporation of new variables, and transfer from multi-equation datasets. Our results demonstrate that advanced neural operator architectures can effectively transfer knowledge across PDE problems.