Solving parametric partial differential equations with deep rectified quadratic unit neural networks
This work provides a theoretical improvement for solving parametric PDEs, which is important for computational science and engineering, but it is incremental as it builds directly on prior ReLU-based methods.
The paper tackles the problem of approximating solution maps for parametric partial differential equations (PDEs) using deep rectified quadratic unit (ReQU) neural networks, deriving an improved upper bound of O(d^3 log^q log(1/ε)) on network size for accuracy ε, compared to O(d^3 log^q(1/ε)) for ReLU networks, and verifies this with numerical experiments.
Implementing deep neural networks for learning the solution maps of parametric partial differential equations (PDEs) turns out to be more efficient than using many conventional numerical methods. However, limited theoretical analyses have been conducted on this approach. In this study, we investigate the expressive power of deep rectified quadratic unit (ReQU) neural networks for approximating the solution maps of parametric PDEs. The proposed approach is motivated by the recent important work of G. Kutyniok, P. Petersen, M. Raslan and R. Schneider (Gitta Kutyniok, Philipp Petersen, Mones Raslan, and Reinhold Schneider. A theoretical analysis of deep neural networks and parametric pdes. Constructive Approximation, pages 1-53, 2021), which uses deep rectified linear unit (ReLU) neural networks for solving parametric PDEs. In contrast to the previously established complexity-bound $\mathcal{O}\left(d^3\log_{2}^{q}(1/ ε) \right)$ for ReLU neural networks, we derive an upper bound $\mathcal{O}\left(d^3\log_{2}^{q}\log_{2}(1/ ε) \right)$ on the size of the deep ReQU neural network required to achieve accuracy $ε>0$, where $d$ is the dimension of reduced basis representing the solutions. Our method takes full advantage of the inherent low-dimensionality of the solution manifolds and better approximation performance of deep ReQU neural networks. Numerical experiments are performed to verify our theoretical result.