Enhancing training of physics-informed neural networks using domain-decomposition based preconditioning strategies
This work addresses training inefficiencies in PINNs for computational physics, offering incremental improvements through novel preconditioning methods.
The authors tackled the slow convergence of physics-informed neural networks (PINNs) by introducing nonlinear additive and multiplicative preconditioning strategies based on domain decomposition, which significantly improved convergence and accuracy in solving partial differential equations.
We propose to enhance the training of physics-informed neural networks (PINNs). To this aim, we introduce nonlinear additive and multiplicative preconditioning strategies for the widely used L-BFGS optimizer. The nonlinear preconditioners are constructed by utilizing the Schwarz domain-decomposition framework, where the parameters of the network are decomposed in a layer-wise manner. Through a series of numerical experiments, we demonstrate that both, additive and multiplicative preconditioners significantly improve the convergence of the standard L-BFGS optimizer, while providing more accurate solutions of the underlying partial differential equations. Moreover, the additive preconditioner is inherently parallel, thus giving rise to a novel approach to model parallelism.