Pretraining Codomain Attention Neural Operators for Solving Multiphysics PDEs
This addresses the problem of limited data and complex interactions in multiphysics simulations for researchers and engineers, representing a novel method rather than an incremental improvement.
The paper tackles the challenge of solving multiphysics PDEs with coupled equations by proposing CoDA-NO, a neural operator that tokenizes functions along the codomain for pretraining, and it outperforms existing methods by over 36% on tasks like fluid flow simulations.
Existing neural operator architectures face challenges when solving multiphysics problems with coupled partial differential equations (PDEs) due to complex geometries, interactions between physical variables, and the limited amounts of high-resolution training data. To address these issues, we propose Codomain Attention Neural Operator (CoDA-NO), which tokenizes functions along the codomain or channel space, enabling self-supervised learning or pretraining of multiple PDE systems. Specifically, we extend positional encoding, self-attention, and normalization layers to function spaces. CoDA-NO can learn representations of different PDE systems with a single model. We evaluate CoDA-NO's potential as a backbone for learning multiphysics PDEs over multiple systems by considering few-shot learning settings. On complex downstream tasks with limited data, such as fluid flow simulations, fluid-structure interactions, and Rayleigh-Bénard convection, we found CoDA-NO to outperform existing methods by over 36%.