Optimization and generalization analysis for two-layer physics-informed neural networks without over-parametrization
This work addresses computational efficiency for researchers using PINNs in scientific computing, though it is incremental as it builds on prior over-parameterization analyses.
The authors tackled the issue of prohibitive computational costs in physics-informed neural networks (PINNs) by analyzing stochastic gradient descent (SGD) for two-layer PINNs without over-parameterization, showing that with a width threshold depending on ε, both training and expected losses decrease below O(ε).
This work focuses on the behavior of stochastic gradient descent (SGD) in solving least-squares regression with physics-informed neural networks (PINNs). Past work on this topic has been based on the over-parameterization regime, whose convergence may require the network width to increase vastly with the number of training samples. So, the theory derived from over-parameterization may incur prohibitive computational costs and is far from practical experiments. We perform new optimization and generalization analysis for SGD in training two-layer PINNs, making certain assumptions about the target function to avoid over-parameterization. Given $ε>0$, we show that if the network width exceeds a threshold that depends only on $ε$ and the problem, then the training loss and expected loss will decrease below $O(ε)$.