Beyond Derivative Pathology of PINNs: Variable Splitting Strategy with Convergence Analysis
This work addresses a fundamental convergence issue in PINNs, which is significant for researchers and practitioners relying on PINNs for solving PDEs, offering a theoretical guarantee for a subset of problems.
This paper identifies a fundamental issue in Physics-informed neural networks (PINNs) where minimizing the loss function does not guarantee convergence to a PDE solution, attributing this to unregulated derivative behavior. They propose a variable splitting strategy that parameterizes the solution's gradient as an auxiliary variable, which allows direct monitoring and regulation of the gradient. This method is proven to guarantee convergence to a generalized solution for second-order linear PDEs.
Physics-informed neural networks (PINNs) have recently emerged as effective methods for solving partial differential equations (PDEs) in various problems. Substantial research focuses on the failure modes of PINNs due to their frequent inaccuracies in predictions. However, most are based on the premise that minimizing the loss function to zero causes the network to converge to a solution of the governing PDE. In this study, we prove that PINNs encounter a fundamental issue that the premise is invalid. We also reveal that this issue stems from the inability to regulate the behavior of the derivatives of the predicted solution. Inspired by the \textit{derivative pathology} of PINNs, we propose a \textit{variable splitting} strategy that addresses this issue by parameterizing the gradient of the solution as an auxiliary variable. We demonstrate that using the auxiliary variable eludes derivative pathology by enabling direct monitoring and regulation of the gradient of the predicted solution. Moreover, we prove that the proposed method guarantees convergence to a generalized solution for second-order linear PDEs, indicating its applicability to various problems.