Provable approximation properties for deep neural networks
This provides theoretical guarantees for neural network approximation on manifolds, which is incremental as it builds on existing wavelet and ReLU-based methods.
The paper tackles the problem of approximating functions on manifolds using deep neural networks by constructing a sparsely-connected depth-4 network and bounding its approximation error, with network size depending on manifold dimension, curvature, and function complexity rather than ambient dimension.
We discuss approximation of functions using deep neural nets. Given a function $f$ on a $d$-dimensional manifold $Γ\subset \mathbb{R}^m$, we construct a sparsely-connected depth-4 neural network and bound its error in approximating $f$. The size of the network depends on dimension and curvature of the manifold $Γ$, the complexity of $f$, in terms of its wavelet description, and only weakly on the ambient dimension $m$. Essentially, our network computes wavelet functions, which are computed from Rectified Linear Units (ReLU)