Error bounds for approximations with deep ReLU neural networks in $W^{s,p}$ norms
This work extends approximation theory for ReLU networks to regimes relevant for numerical analysis of partial differential equations, addressing a domain-specific problem.
The paper tackles the problem of approximating Sobolev-regular functions using deep ReLU neural networks in weaker Sobolev norms, constructing networks that achieve specific approximation rates and establishing lower bounds for such approximations.
We analyze approximation rates of deep ReLU neural networks for Sobolev-regular functions with respect to weaker Sobolev norms. First, we construct, based on a calculus of ReLU networks, artificial neural networks with ReLU activation functions that achieve certain approximation rates. Second, we establish lower bounds for the approximation by ReLU neural networks for classes of Sobolev-regular functions. Our results extend recent advances in the approximation theory of ReLU networks to the regime that is most relevant for applications in the numerical analysis of partial differential equations.