Optimal time sampling in physics-informed neural networks
This work addresses a specific bottleneck in PINNs for scientific computing, offering incremental improvements in sampling efficiency for time-dependent problems.
The paper tackled the problem of optimal time sampling in physics-informed neural networks (PINNs) for time-dependent equations, showing that the optimal sampling follows a truncated exponential distribution and providing conditions for when uniform sampling is best or not, with numerical examples on linear, Burgers', and Lorenz equations.
Physics-informed neural networks (PINN) is a extremely powerful paradigm used to solve equations encountered in scientific computing applications. An important part of the procedure is the minimization of the equation residual which includes, when the equation is time-dependent, a time sampling. It was argued in the literature that the sampling need not be uniform but should overweight initial time instants, but no rigorous explanation was provided for this choice. In the present work we take some prototypical examples and, under standard hypothesis concerning the neural network convergence, we show that the optimal time sampling follows a (truncated) exponential distribution. In particular we explain when is best to use uniform time sampling and when one should not. The findings are illustrated with numerical examples on linear equation, Burgers' equation and the Lorenz system.