Accelerating Particle-based Energetic Variational Inference
This work addresses a bottleneck in machine learning for practitioners using particle-based inference, offering an incremental improvement over prior methods.
The paper tackles the computational inefficiency of particle-based variational inference by proposing a novel method that avoids repeated evaluation of inter-particle interaction terms, resulting in significant reductions in computational cost and outperforming existing approaches in efficiency, robustness, and accuracy.
In this work, we propose a novel particle-based variational inference (ParVI) method that accelerates the EVI-Im. Inspired by energy quadratization (EQ) and operator splitting techniques for gradient flows, our approach efficiently drives particles towards the target distribution. Unlike EVI-Im, which employs the implicit Euler method to solve variational-preserving particle dynamics for minimizing the KL divergence, derived using a "discretize-then-variational" approach, the proposed algorithm avoids repeated evaluation of inter-particle interaction terms, significantly reducing computational cost. The framework is also extensible to other gradient-based sampling techniques. Through several numerical experiments, we demonstrate that our method outperforms existing ParVI approaches in efficiency, robustness, and accuracy.