Condensed Stein Variational Gradient Descent for Uncertainty Quantification of Neural Networks
This work addresses uncertainty quantification for neural network parameters in domains like solid mechanics, but it appears incremental as it builds on existing Stein variational gradient descent methods.
The authors tackled the problem of uncertainty quantification in neural networks by proposing a condensed Stein variational gradient descent method that concurrently sparsifies, trains, and quantifies uncertainty on parameters, not just outputs, demonstrating it with an illustrative example and a solid mechanics application.
We propose a Stein variational gradient descent method to concurrently sparsify, train, and provide uncertainty quantification of a complexly parameterized model such as a neural network. It employs a graph reconciliation and condensation process to reduce complexity and increase similarity in the Stein ensemble of parameterizations. Therefore, the proposed condensed Stein variational gradient (cSVGD) method provides uncertainty quantification on parameters, not just outputs. Furthermore, the parameter reduction speeds up the convergence of the Stein gradient descent as it reduces the combinatorial complexity by aligning and differentiating the sensitivity to parameters. These properties are demonstrated with an illustrative example and an application to a representation problem in solid mechanics.