Shallow neural network representation of polynomials
This addresses the approximation capabilities of shallow neural networks for polynomial and smooth functions, with implications for theoretical machine learning.
The paper proves that d-variate polynomials of degree R can be represented by shallow neural networks with width 2(R+d)^d, and uses this to derive minimax optimal convergence rates, up to a logarithmic factor, for shallow networks in univariate regression.
We show that $d$-variate polynomials of degree $R$ can be represented on $[0,1]^d$ as shallow neural networks of width $2(R+d)^d$. Also, by SNN representation of localized Taylor polynomials of univariate $C^β$-smooth functions, we derive for shallow networks the minimax optimal rate of convergence, up to a logarithmic factor, to unknown univariate regression function.