Tractable Representation Learning with Probabilistic Circuits
This work addresses a gap in representation learning for probabilistic reasoning, offering a flexible method that could benefit tasks like robust inference and out-of-distribution detection, though it appears incremental as it builds on existing PC frameworks.
The authors tackled the underexplored problem of representation learning with probabilistic circuits by introducing autoencoding probabilistic circuits (APCs), which outperform existing PC-based methods in reconstruction quality and show competitive embeddings with superior robustness in handling missing data compared to neural autoencoders.
Probabilistic circuits (PCs) are powerful probabilistic models that enable exact and tractable inference, making them highly suitable for probabilistic reasoning and inference tasks. While dominant in neural networks, representation learning with PCs remains underexplored, with prior approaches relying on external neural embeddings or activation-based encodings. To address this gap, we introduce autoencoding probabilistic circuits (APCs), a novel framework leveraging the tractability of PCs to model probabilistic embeddings explicitly. APCs extend PCs by jointly modeling data and embeddings, obtaining embedding representations through tractable probabilistic inference. The PC encoder allows the framework to natively handle arbitrary missing data and is seamlessly integrated with a neural decoder in a hybrid, end-to-end trainable architecture enabled by differentiable sampling. Our empirical evaluation demonstrates that APCs outperform existing PC-based autoencoding methods in reconstruction quality, generate embeddings competitive with, and exhibit superior robustness in handling missing data compared to neural autoencoders. These results highlight APCs as a powerful and flexible representation learning method that exploits the probabilistic inference capabilities of PCs, showing promising directions for robust inference, out-of-distribution detection, and knowledge distillation.