Sub-Sequential Physics-Informed Learning with State Space Model
This addresses a critical failure mode in PINNs for solving PDEs, offering a significant improvement for computational physics and engineering applications.
The paper tackles the problem of Physics-Informed Neural Networks (PINNs) failing to propagate initial conditions in PDEs, proposing PINNMamba, a framework using State Space Models for sub-sequence modeling, which reduces errors by up to 86.3% compared to state-of-the-art methods.
Physics-Informed Neural Networks (PINNs) are a kind of deep-learning-based numerical solvers for partial differential equations (PDEs). Existing PINNs often suffer from failure modes of being unable to propagate patterns of initial conditions. We discover that these failure modes are caused by the simplicity bias of neural networks and the mismatch between PDE's continuity and PINN's discrete sampling. We reveal that the State Space Model (SSM) can be a continuous-discrete articulation allowing initial condition propagation, and that simplicity bias can be eliminated by aligning a sequence of moderate granularity. Accordingly, we propose PINNMamba, a novel framework that introduces sub-sequence modeling with SSM. Experimental results show that PINNMamba can reduce errors by up to 86.3\% compared with state-of-the-art architecture. Our code is available at https://github.com/miniHuiHui/PINNMamba.