Domain decomposition architectures and Gauss-Newton training for physics-informed neural networks
This work addresses the difficult training process for neural network-based PDE solvers, which is an incremental improvement for computational science and engineering applications.
The paper tackled the challenge of training physics-informed neural networks for solving partial differential equations by combining domain decomposition architectures with Gauss-Newton optimization, resulting in faster convergence compared to gradient-based methods like Adam, though at the cost of solving ill-conditioned linear systems per iteration.
Approximating the solutions of boundary value problems governed by partial differential equations with neural networks is challenging, largely due to the difficult training process. This difficulty can be partly explained by the spectral bias, that is, the slower convergence of high-frequency components, and can be mitigated by localizing neural networks via (overlapping) domain decomposition. We combine this localization with the Gauss-Newton method as the optimizer to obtain faster convergence than gradient-based schemes such as Adam; this comes at the cost of solving an ill-conditioned linear system in each iteration. Domain decomposition induces a block-sparse structure in the otherwise dense Gauss-Newton system, reducing the computational cost per iteration. Our numerical results indicate that combining localization and Gauss-Newton optimization is promising for neural network-based solvers for partial differential equations.