Neural Networks for Singular Perturbations
This work addresses the challenge of efficiently representing solutions with boundary layers in computational mathematics, offering incremental theoretical guarantees for neural network applications in this domain.
The paper tackles the problem of approximating solutions to singularly perturbed elliptic boundary value problems using deep neural networks, proving robust exponential expression rate bounds in Sobolev norms that are uniform with respect to the perturbation parameter for various architectures like ReLU, spiking, tanh, and sigmoid networks.
We prove deep neural network (DNN for short) expressivity rate bounds for solution sets of a model class of singularly perturbed, elliptic two-point boundary value problems, in Sobolev norms, on the bounded interval $(-1,1)$. We assume that the given source term and reaction coefficient are analytic in $[-1,1]$. We establish expression rate bounds in Sobolev norms in terms of the NN size which are uniform with respect to the singular perturbation parameter for several classes of DNN architectures. In particular, ReLU NNs, spiking NNs, and $\tanh$- and sigmoid-activated NNs. The latter activations can represent ``exponential boundary layer solution features'' explicitly, in the last hidden layer of the DNN, i.e. in a shallow subnetwork, and afford improved robust expression rate bounds in terms of the NN size. We prove that all DNN architectures allow robust exponential solution expression in so-called `energy' as well as in `balanced' Sobolev norms, for analytic input data.