Adversarial Multi-task Learning Enhanced Physics-informed Neural Networks for Solving Partial Differential Equations
This addresses the challenge of improving generalizability for physics-informed neural networks in scientific computation, though it appears to be an incremental improvement over existing PDE-solving neural network approaches.
The paper tackles the problem of neural network performance dropping in high nonlinearity domains when solving partial differential equations (PDEs), and introduces a novel approach using multi-task learning and adversarial training that reduces error on unseen data points compared to previous methods.
Recently, researchers have utilized neural networks to accurately solve partial differential equations (PDEs), enabling the mesh-free method for scientific computation. Unfortunately, the network performance drops when encountering a high nonlinearity domain. To improve the generalizability, we introduce the novel approach of employing multi-task learning techniques, the uncertainty-weighting loss and the gradients surgery, in the context of learning PDE solutions. The multi-task scheme exploits the benefits of learning shared representations, controlled by cross-stitch modules, between multiple related PDEs, which are obtainable by varying the PDE parameterization coefficients, to generalize better on the original PDE. Encouraging the network pay closer attention to the high nonlinearity domain regions that are more challenging to learn, we also propose adversarial training for generating supplementary high-loss samples, similarly distributed to the original training distribution. In the experiments, our proposed methods are found to be effective and reduce the error on the unseen data points as compared to the previous approaches in various PDE examples, including high-dimensional stochastic PDEs.