Rethinking Variational Inference for Probabilistic Programs with Stochastic Support
This addresses a bottleneck in probabilistic programming for researchers and practitioners, offering a novel method rather than an incremental improvement.
The paper tackles the problem of variational inference for probabilistic programs with stochastic support by introducing Support Decomposition Variational Inference (SDVI), which decomposes programs into sub-programs with static support and builds separate guides, resulting in substantial improvements in inference performance.
We introduce Support Decomposition Variational Inference (SDVI), a new variational inference (VI) approach for probabilistic programs with stochastic support. Existing approaches to this problem rely on designing a single global variational guide on a variable-by-variable basis, while maintaining the stochastic control flow of the original program. SDVI instead breaks the program down into sub-programs with static support, before automatically building separate sub-guides for each. This decomposition significantly aids in the construction of suitable variational families, enabling, in turn, substantial improvements in inference performance.