LF-PPL: A Low-Level First Order Probabilistic Programming Language for Non-Differentiable Models
This work addresses a bottleneck in probabilistic programming for researchers and practitioners dealing with non-differentiable models, offering an incremental improvement by enabling gradient-based inference in previously challenging scenarios.
The authors tackled the problem of performing efficient inference in probabilistic models with non-differentiable components, such as those involving continuous, discrete, or piecewise-continuous variables, by developing LF-PPL, a low-level first-order probabilistic programming language. They demonstrated its effectiveness by incorporating a discontinuous Hamiltonian Monte Carlo (DHMC) inference engine that automates and improves efficiency for such models.
We develop a new Low-level, First-order Probabilistic Programming Language (LF-PPL) suited for models containing a mix of continuous, discrete, and/or piecewise-continuous variables. The key success of this language and its compilation scheme is in its ability to automatically distinguish parameters the density function is discontinuous with respect to, while further providing runtime checks for boundary crossings. This enables the introduction of new inference engines that are able to exploit gradient information, while remaining efficient for models which are not everywhere differentiable. We demonstrate this ability by incorporating a discontinuous Hamiltonian Monte Carlo (DHMC) inference engine that is able to deliver automated and efficient inference for non-differentiable models. Our system is backed up by a mathematical formalism that ensures that any model expressed in this language has a density with measure zero discontinuities to maintain the validity of the inference engine.