Multi-Preconditioned LBFGS for Training Finite-Basis PINNs
This work addresses training efficiency for domain-decomposed physics-informed neural networks, representing an incremental improvement with specific gains in speed and accuracy.
The paper tackles the problem of slow convergence and high communication overhead in training finite-basis physics-informed neural networks (FBPINNs) by introducing a multi-preconditioned LBFGS algorithm, which improves convergence speed and model accuracy over standard LBFGS.
A multi-preconditioned LBFGS (MP-LBFGS) algorithm is introduced for training finite-basis physics-informed neural networks (FBPINNs). The algorithm is motivated by the nonlinear additive Schwarz method and exploits the domain-decomposition-inspired additive architecture of FBPINNs, in which local neural networks are defined on subdomains, thereby localizing the network representation. Parallel, subdomain-local quasi-Newton corrections are then constructed on the corresponding local parts of the architecture. A key feature is a novel nonlinear multi-preconditioning mechanism, in which subdomain corrections are optimally combined through the solution of a low-dimensional subspace minimization problem. Numerical experiments indicate that MP-LBFGS can improve convergence speed, as well as model accuracy over standard LBFGS while incurring lower communication overhead.