GAD-PVI: A General Accelerated Dynamic-Weight Particle-Based Variational Inference Framework
This work addresses a bottleneck in variational inference for machine learning practitioners, offering incremental improvements in efficiency and accuracy.
The paper tackled the problem of improving Particle-based Variational Inference (ParVI) by proposing GAD-PVI, a framework that combines accelerated position updates and dynamic weight adjustment, resulting in faster convergence and reduced approximation error compared to state-of-the-art methods.
Particle-based Variational Inference (ParVI) methods approximate the target distribution by iteratively evolving finite weighted particle systems. Recent advances of ParVI methods reveal the benefits of accelerated position update strategies and dynamic weight adjustment approaches. In this paper, we propose the first ParVI framework that possesses both accelerated position update and dynamical weight adjustment simultaneously, named the General Accelerated Dynamic-Weight Particle-based Variational Inference (GAD-PVI) framework. Generally, GAD-PVI simulates the semi-Hamiltonian gradient flow on a novel Information-Fisher-Rao space, which yields an additional decrease on the local functional dissipation. GAD-PVI is compatible with different dissimilarity functionals and associated smoothing approaches under three information metrics. Experiments on both synthetic and real-world data demonstrate the faster convergence and reduced approximation error of GAD-PVI methods over the state-of-the-art.