Limitations on approximation by deep and shallow neural networks
This work addresses theoretical limitations in neural network approximation for researchers in machine learning and approximation theory, but it is incremental as it builds on existing concepts like Lipschitz widths.
The paper tackles the problem of approximating compact sets of functions using deep and shallow neural networks by proving Carl's type inequalities, which provide lower bounds on approximation errors. The results are derived from the study of Lipschitz widths, offering theoretical limitations on how well these networks can approximate functions.
We prove Carl's type inequalities for the error of approximation of compact sets K by deep and shallow neural networks. This in turn gives lower bounds on how well we can approximate the functions in K when requiring the approximants to come from outputs of such networks. Our results are obtained as a byproduct of the study of the recently introduced Lipschitz widths.