Coupling parameter and particle dynamics for adaptive sampling in Neural Galerkin schemes
This addresses inefficiencies in neural network-based PDE solvers for researchers in computational science, though it is incremental as it builds on existing adaptive sampling methods.
The paper tackles the challenge of efficiently training neural networks to solve partial differential equations by introducing Neural Galerkin schemes that use adaptive sampling from particle ensembles to estimate training loss, reducing variance and enabling accurate results with few particles even in high-dimensional problems.
Training nonlinear parametrizations such as deep neural networks to numerically approximate solutions of partial differential equations is often based on minimizing a loss that includes the residual, which is analytically available in limited settings only. At the same time, empirically estimating the training loss is challenging because residuals and related quantities can have high variance, especially for transport-dominated and high-dimensional problems that exhibit local features such as waves and coherent structures. Thus, estimators based on data samples from un-informed, uniform distributions are inefficient. This work introduces Neural Galerkin schemes that estimate the training loss with data from adaptive distributions, which are empirically represented via ensembles of particles. The ensembles are actively adapted by evolving the particles with dynamics coupled to the nonlinear parametrizations of the solution fields so that the ensembles remain informative for estimating the training loss. Numerical experiments indicate that few dynamic particles are sufficient for obtaining accurate empirical estimates of the training loss, even for problems with local features and with high-dimensional spatial domains.