Two-Layer Neural Networks for Partial Differential Equations: Optimization and Generalization Theory
This provides theoretical guarantees for neural network-based PDE solvers, which is incremental but important for computational science and engineering.
The paper tackles solving second-order linear PDEs using two-layer neural networks via least-squares optimization, showing that gradient descent finds a global minimizer under over-parametrization and analyzing generalization error with regularization in Barron-type spaces.
The problem of solving partial differential equations (PDEs) can be formulated into a least-squares minimization problem, where neural networks are used to parametrize PDE solutions. A global minimizer corresponds to a neural network that solves the given PDE. In this paper, we show that the gradient descent method can identify a global minimizer of the least-squares optimization for solving second-order linear PDEs with two-layer neural networks under the assumption of over-parametrization. We also analyze the generalization error of the least-squares optimization for second-order linear PDEs and two-layer neural networks, when the right-hand-side function of the PDE is in a Barron-type space and the least-squares optimization is regularized with a Barron-type norm, without the over-parametrization assumption.