Functional Tensors for Probabilistic Programming
This work addresses the problem of creating flexible probabilistic programming frameworks for researchers and practitioners, representing an incremental advancement by building on existing tensor concepts.
The paper tackles the challenge of designing versatile probabilistic programming systems by introducing functional tensors, a software abstraction that unifies modeling and inference strategies, and demonstrates its integration into Pyro to enable a wide variety of inference methods, including mixed exact and approximate approaches.
It is a significant challenge to design probabilistic programming systems that can accommodate a wide variety of inference strategies within a unified framework. Noting that the versatility of modern automatic differentiation frameworks is based in large part on the unifying concept of tensors, we describe a software abstraction for integration --functional tensors-- that captures many of the benefits of tensors, while also being able to describe continuous probability distributions. Moreover, functional tensors are a natural candidate for generalized variable elimination and parallel-scan filtering algorithms that enable parallel exact inference for a large family of tractable modeling motifs. We demonstrate the versatility of functional tensors by integrating them into the modeling frontend and inference backend of the Pyro programming language. In experiments we show that the resulting framework enables a large variety of inference strategies, including those that mix exact and approximate inference.