Higher-order Quasi-Monte Carlo Training of Deep Neural Networks
This addresses efficient surrogate modeling for engineering design problems with uncertain inputs, offering a theoretically-grounded incremental improvement over existing training methods.
The paper tackles training deep neural networks as surrogates for engineering design maps by developing a novel Quasi-Monte Carlo method with deterministic training points, proving it achieves higher-order error decay without the curse of dimensionality under certain conditions, and validating this with numerical experiments on PDE-based maps.
We present a novel algorithmic approach and an error analysis leveraging Quasi-Monte Carlo points for training deep neural network (DNN) surrogates of Data-to-Observable (DtO) maps in engineering design. Our analysis reveals higher-order consistent, deterministic choices of training points in the input data space for deep and shallow Neural Networks with holomorphic activation functions such as tanh. These novel training points are proved to facilitate higher-order decay (in terms of the number of training samples) of the underlying generalization error, with consistency error bounds that are free from the curse of dimensionality in the input data space, provided that DNN weights in hidden layers satisfy certain summability conditions. We present numerical experiments for DtO maps from elliptic and parabolic PDEs with uncertain inputs that confirm the theoretical analysis.