Analysis of Deep Neural Networks with Quasi-optimal polynomial approximation rates
This provides a theoretical foundation for efficient function approximation in machine learning, though it is incremental as it builds on existing polynomial approximation methods.
The paper tackles the problem of approximating high-dimensional functions with deep neural networks, achieving a sub-exponential error rate in the number of polynomial functions used, with network complexity that is algebraic in this number.
We show the existence of a deep neural network capable of approximating a wide class of high-dimensional approximations. The construction of the proposed neural network is based on a quasi-optimal polynomial approximation. We show that this network achieves an error rate that is sub-exponential in the number of polynomial functions, $M$, used in the polynomial approximation. The complexity of the network which achieves this sub-exponential rate is shown to be algebraic in $M$.