Mitigating Gradient Pathology in PINNs through Aligned Constraint
For researchers using PINNs to solve PDEs, this work addresses a fundamental training instability, though it is an incremental improvement over existing adaptive weighting and hard constraint methods.
The paper identifies gradient pathology in PINNs where PDE and boundary gradients conflict, and proposes CAML (Constraint-Aligned loss with Manifold Lifting) to mitigate this by reformulating constraints and adding a delay factor, achieving significant improvements in numerical stability and efficiency on complex PDE problems.
While Physics-Informed Neural Networks (PINNs) are powerful for solving Partial Differential Equations (PDEs), their training is often paralyzed by gradient pathology. The gradients from the PDE residuals and boundary constraints oppose each other, trapping the model in local minima. Current solutions, such as adaptive weighting or hard constraints, either fail to fundamentally resolve this ill-conditioning or are limited to simple geometries. In this study, we systematically analyze the possible causes of this gradient pathology from the perspectives of loss landscapes and optimization dynamics. Based on the obtained conclusion, we propose Constraint-Aligned loss with Manifold Lifting (CAML). By reformulating all zeroth-order terms into aligned constraints, our method effectively mitigates gradient conflicts. In addition, we introduce a delay factor to help the optimizer skip the high-curvature area. Experiments demonstrate that our CAML significantly enhances numerical stability and efficiency in highly complex PINN problems. Our code is open-sourced on https://github.com/YichenLuo-0/CAML.