Diffusion models for probabilistic programming
This work addresses inference challenges in probabilistic programming for users of Bayesian modeling, offering a practical improvement over existing methods.
The authors tackled the problem of automated approximate inference in probabilistic programming languages by proposing Diffusion Model Variational Inference (DMVI), which uses diffusion models as variational approximations and shows more accurate posterior inferences than contemporary methods with similar computational cost and less manual tuning.
We propose Diffusion Model Variational Inference (DMVI), a novel method for automated approximate inference in probabilistic programming languages (PPLs). DMVI utilizes diffusion models as variational approximations to the true posterior distribution by deriving a novel bound to the marginal likelihood objective used in Bayesian modelling. DMVI is easy to implement, allows hassle-free inference in PPLs without the drawbacks of, e.g., variational inference using normalizing flows, and does not make any constraints on the underlying neural network model. We evaluate DMVI on a set of common Bayesian models and show that its posterior inferences are in general more accurate than those of contemporary methods used in PPLs while having a similar computational cost and requiring less manual tuning.