Distributionally robust approximation property of neural networks
This provides a theoretical foundation for neural networks in distributionally robust settings, which is incremental but important for reliability in applications like uncertainty quantification.
The paper establishes that several neural network architectures, including feedforward networks with non-polynomial activations and deep narrow ReLU networks, possess a universal approximation property uniformly across weakly compact families of measures, extending classical results beyond the L^p-setting.
The universal approximation property uniformly with respect to weakly compact families of measures is established for several classes of neural networks. To that end, we prove that these neural networks are dense in Orlicz spaces, thereby extending classical universal approximation theorems even beyond the traditional $L^p$-setting. The covered classes of neural networks include widely used architectures like feedforward neural networks with non-polynomial activation functions, deep narrow networks with ReLU activation functions and functional input neural networks.