Bayesian Structure Scores for Probabilistic Circuits
This work addresses a bottleneck in probabilistic modeling for researchers and practitioners by offering a more rigorous approach to structure learning in PCs, though it is incremental as it adapts known Bayesian methods from graphical models to this domain.
The paper tackles the problem of structure learning in probabilistic circuits (PCs) by developing Bayesian structure scores, which provide a principled objective to replace heuristic methods, resulting in a fast and hyper-parameter-free learner that achieves good trade-offs in training time and model fit, as measured by log-likelihood.
Probabilistic circuits (PCs) are a prominent representation of probability distributions with tractable inference. While parameter learning in PCs is rigorously studied, structure learning is often more based on heuristics than on principled objectives. In this paper, we develop Bayesian structure scores for deterministic PCs, i.e., the structure likelihood with parameters marginalized out, which are well known as rigorous objectives for structure learning in probabilistic graphical models. When used within a greedy cutset algorithm, our scores effectively protect against overfitting and yield a fast and almost hyper-parameter-free structure learner, distinguishing it from previous approaches. In experiments, we achieve good trade-offs between training time and model fit in terms of log-likelihood. Moreover, the principled nature of Bayesian scores unlocks PCs for accommodating frameworks such as structural expectation-maximization.