Accelerating Convergence of Stein Variational Gradient Descent via Deep Unfolding
This work addresses convergence speed for users of SVGD in Bayesian inference, but it is incremental as it builds on existing SVGD methods with a deep learning enhancement.
The authors tackled the slow convergence of Stein Variational Gradient Descent (SVGD) by integrating deep unfolding to learn internal parameters, resulting in faster convergence across tasks like Gaussian mixture sampling and Bayesian logistic regression.
Stein variational gradient descent (SVGD) is a prominent particle-based variational inference method used for sampling a target distribution. SVGD has attracted interest for application in machine-learning techniques such as Bayesian inference. In this paper, we propose novel trainable algorithms that incorporate a deep-learning technique called deep unfolding,into SVGD. This approach facilitates the learning of the internal parameters of SVGD, thereby accelerating its convergence speed. To evaluate the proposed trainable SVGD algorithms, we conducted numerical simulations of three tasks: sampling a one-dimensional Gaussian mixture, performing Bayesian logistic regression, and learning Bayesian neural networks. The results show that our proposed algorithms exhibit faster convergence than the conventional variants of SVGD.