Building Expressive and Tractable Probabilistic Generative Models: A Review
This is an incremental survey for researchers in machine learning, summarizing existing work without new results.
The paper reviews advancements in tractable probabilistic generative models, focusing on Probabilistic Circuits (PCs) to address trade-offs between expressivity and tractability, and provides a taxonomy and discussion of future challenges.
We present a comprehensive survey of the advancements and techniques in the field of tractable probabilistic generative modeling, primarily focusing on Probabilistic Circuits (PCs). We provide a unified perspective on the inherent trade-offs between expressivity and tractability, highlighting the design principles and algorithmic extensions that have enabled building expressive and efficient PCs, and provide a taxonomy of the field. We also discuss recent efforts to build deep and hybrid PCs by fusing notions from deep neural models, and outline the challenges and open questions that can guide future research in this evolving field.