TSONN: Time-stepping-oriented neural network for solving partial differential equations
This addresses a key bottleneck for researchers and practitioners using neural networks to solve PDEs, offering a robust optimization-based solver that improves convergence and stability, though it is incremental as it builds on existing PINN methods.
The paper tackles the challenge of unstable training and incorrect results in physics-informed neural networks (PINNs) for solving partial differential equations (PDEs) by integrating time-stepping methods with deep learning, transforming the optimization into well-conditioned sub-problems. The result shows stable training and correct results in problems where standard PINNs fail, with explicit schemes allowing significantly larger time steps and implicit schemes being straightforward to implement.
Deep neural networks (DNNs), especially physics-informed neural networks (PINNs), have recently become a new popular method for solving forward and inverse problems governed by partial differential equations (PDEs). However, these methods still face challenges in achieving stable training and obtaining correct results in many problems, since minimizing PDE residuals with PDE-based soft constraint make the problem ill-conditioned. Different from all existing methods that directly minimize PDE residuals, this work integrates time-stepping method with deep learning, and transforms the original ill-conditioned optimization problem into a series of well-conditioned sub-problems over given pseudo time intervals. The convergence of model training is significantly improved by following the trajectory of the pseudo time-stepping process, yielding a robust optimization-based PDE solver. Our results show that the proposed method achieves stable training and correct results in many problems that standard PINNs fail to solve, requiring only a simple modification on the loss function. In addition, we demonstrate several novel properties and advantages of time-stepping methods within the framework of neural network-based optimization approach, in comparison to traditional grid-based numerical method. Specifically, explicit scheme allows significantly larger time step, while implicit scheme can be implemented as straightforwardly as explicit scheme.