Improving the performance of Stein variational inference through extreme sparsification of physically-constrained neural network models
This addresses uncertainty quantification for scientific machine learning models with many parameters, offering an incremental improvement over prior Stein variational inference methods.
The paper tackles the curse of dimensionality in uncertainty quantification for scientific machine learning by introducing $L_0$ sparsification prior to Stein variational gradient descent ($L_0$+SVGD), showing it is more robust and efficient than existing methods, with faster convergence and better performance in noisy and extrapolated scenarios.
Most scientific machine learning (SciML) applications of neural networks involve hundreds to thousands of parameters, and hence, uncertainty quantification for such models is plagued by the curse of dimensionality. Using physical applications, we show that $L_0$ sparsification prior to Stein variational gradient descent ($L_0$+SVGD) is a more robust and efficient means of uncertainty quantification, in terms of computational cost and performance than the direct application of SGVD or projected SGVD methods. Specifically, $L_0$+SVGD demonstrates superior resilience to noise, the ability to perform well in extrapolated regions, and a faster convergence rate to an optimal solution.