Discrete-Continuous Mixtures in Probabilistic Programming: Generalized Semantics and Inference Algorithms
This addresses a fundamental limitation in probabilistic programming for AI applications, enabling more flexible modeling of complex distributions.
The paper tackles the limited support for discrete-continuous mixture distributions in probabilistic programming languages by developing measure-theoretic Bayesian networks (MTBNs) to provide more general semantics and two new provably correct sampling algorithms (LLW and LPF). The authors integrate MTBNs into the BLOG system and demonstrate effectiveness through representative examples.
Despite the recent successes of probabilistic programming languages (PPLs) in AI applications, PPLs offer only limited support for random variables whose distributions combine discrete and continuous elements. We develop the notion of measure-theoretic Bayesian networks (MTBNs) and use it to provide more general semantics for PPLs with arbitrarily many random variables defined over arbitrary measure spaces. We develop two new general sampling algorithms that are provably correct under the MTBN framework: the lexicographic likelihood weighting (LLW) for general MTBNs and the lexicographic particle filter (LPF), a specialized algorithm for state-space models. We further integrate MTBNs into a widely used PPL system, BLOG, and verify the effectiveness of the new inference algorithms through representative examples.