A Residual Guided strategy with Generative Adversarial Networks in training Physics-Informed Transformer Networks
This work addresses challenges in physics-driven modeling for multiscale and time-dependent PDE systems, offering a robust solution with significant accuracy improvements.
The paper tackled the problem of unresolved residuals and temporal causality violations in Physics-Informed Neural Networks (PINNs) for nonlinear PDEs, proposing a Residual Guided Training strategy with GANs and Transformers that achieved relative MSE reductions of up to three orders of magnitude on benchmark equations.
Nonlinear partial differential equations (PDEs) are pivotal in modeling complex physical systems, yet traditional Physics-Informed Neural Networks (PINNs) often struggle with unresolved residuals in critical spatiotemporal regions and violations of temporal causality. To address these limitations, we propose a novel Residual Guided Training strategy for Physics-Informed Transformer via Generative Adversarial Networks (GAN). Our framework integrates a decoder-only Transformer to inherently capture temporal correlations through autoregressive processing, coupled with a residual-aware GAN that dynamically identifies and prioritizes high-residual regions. By introducing a causal penalty term and an adaptive sampling mechanism, the method enforces temporal causality while refining accuracy in problematic domains. Extensive numerical experiments on the Allen-Cahn, Klein-Gordon, and Navier-Stokes equations demonstrate significant improvements, achieving relative MSE reductions of up to three orders of magnitude compared to baseline methods. This work bridges the gap between deep learning and physics-driven modeling, offering a robust solution for multiscale and time-dependent PDE systems.