Deep Neural Networks with General Activations: Super-Convergence in Sobolev Norms
It provides a unified theoretical foundation for neural-network-based approaches to PDEs in scientific computing, addressing a significant gap in error-estimation theory.
This paper tackles the problem of approximating weak solutions of partial differential equations (PDEs) using deep neural networks with general activation functions, achieving super-convergence rates in Sobolev norms that surpass classical numerical methods like finite element and spectral methods.
This paper establishes a comprehensive approximation result for deep fully-connected neural networks with commonly-used and general activation functions in Sobolev spaces $W^{n,\infty}$, with errors measured in the $W^{m,p}$-norm for $m < n$ and $1\le p \le \infty$. The derived rates surpass those of classical numerical approximation techniques, such as finite element and spectral methods, exhibiting a phenomenon we refer to as \emph{super-convergence}. Our analysis shows that deep networks with general activations can approximate weak solutions of partial differential equations (PDEs) with superior accuracy compared to traditional numerical methods at the approximation level. Furthermore, this work closes a significant gap in the error-estimation theory for neural-network-based approaches to PDEs, offering a unified theoretical foundation for their use in scientific computing.