R-ParVI: Particle-based variational inference through lens of rewards
This provides a new sampling method for probabilistic models in Bayesian inference and generative modeling, though it appears incremental as it builds on existing particle-based variational inference approaches.
The paper tackles the problem of sampling from partially known densities, such as those in Bayesian inference, by proposing R-ParVI, a reward-guided, gradient-free particle-based variational inference method that approximates target distributions through particle flow driven by rewards, offering fast, flexible, scalable, and stochastic sampling.
A reward-guided, gradient-free ParVI method, \textit{R-ParVI}, is proposed for sampling partially known densities (e.g. up to a constant). R-ParVI formulates the sampling problem as particle flow driven by rewards: particles are drawn from a prior distribution, navigate through parameter space with movements determined by a reward mechanism blending assessments from the target density, with the steady state particle configuration approximating the target geometry. Particle-environment interactions are simulated by stochastic perturbations and the reward mechanism, which drive particles towards high density regions while maintaining diversity (e.g. preventing from collapsing into clusters). R-ParVI offers fast, flexible, scalable and stochastic sampling and inference for a class of probabilistic models such as those encountered in Bayesian inference and generative modelling.