Deep neural network approximation of composite functions without the curse of dimensionality
This addresses the curse of dimensionality in neural network approximation for high-dimensional functions, which is foundational for machine learning and AI applications.
The authors tackled the problem of approximating high-dimensional composite functions with deep neural networks, showing that the number of parameters grows polynomially with input dimension and error, avoiding exponential growth.
In this article we identify a general class of high-dimensional continuous functions that can be approximated by deep neural networks (DNNs) with the rectified linear unit (ReLU) activation without the curse of dimensionality. In other words, the number of DNN parameters grows at most polynomially in the input dimension and the approximation error. The functions in our class can be expressed as a potentially unbounded number of compositions of special functions which include products, maxima, and certain parallelized Lipschitz continuous functions.