PINN Balls: Scaling Second-Order Methods for PINNs with Domain Decomposition and Adaptive Sampling
This addresses a scalability bottleneck for researchers and practitioners using PINNs to solve partial differential equations, representing an incremental improvement with specific gains.
The paper tackles the problem of scaling second-order methods for Physics-Informed Neural Networks (PINNs), which suffer from high memory requirements, by introducing PINN Balls, a model that uses a local Mixture of Experts and adaptive domain decomposition to achieve better accuracy than state-of-the-art methods while maintaining scalability.
Recent advances in Scientific Machine Learning have shown that second-order methods can enhance the training of Physics-Informed Neural Networks (PINNs), making them a suitable alternative to traditional numerical methods for Partial Differential Equations (PDEs). However, second-order methods induce large memory requirements, making them scale poorly with the model size. In this paper, we define a local Mixture of Experts (MoE) combining the parameter-efficiency of ensemble models and sparse coding to enable the use of second-order training. Our model -- \textsc{PINN Balls} -- also features a fully learnable domain decomposition structure, achieved through the use of Adversarial Adaptive Sampling (AAS), which adapts the DD to the PDE and its domain. \textsc{PINN Balls} achieves better accuracy than the state-of-the-art in scientific machine learning, while maintaining invaluable scalability properties and drawing from a sound theoretical background.