Explicit integral representations and quantitative bounds for two-layer ReLU networks
Provides theoretical approximation guarantees for ReLU networks, relevant for understanding their expressivity and generalization.
The paper presents explicit integral representations for two-layer ReLU networks that can represent any multivariate polynomial, and provides quantitative approximation bounds showing that L2 errors depend on monomial coefficients and data distribution rather than dimension or degree.
An approach to construct explicit integral representations for two-layer ReLU networks is presented, which provides relatively simple representations for any multivariate polynomial. Quantitative bounds are provided for a particular, sharpened ReLU integral representation, which involves a harmonic extension and a projection. The bounds demonstrate that functions can be approximated with $L^{2}(\mathcal{D})$ errors that do not depend explicitly on dimension or degree, but rather the coefficients of their monomial expansions and the distribution $\mathcal{D}$.