Probabilistic Circuits with Constraints via Convex Optimization
This addresses the problem of incorporating logical constraints into tractable probabilistic models for researchers and practitioners in machine learning, representing an incremental advancement in hybridizing logic and deep probabilistic models.
This work tackles the problem of integrating probabilistic propositional logic constraints into probabilistic circuits (PCs) by developing an approach that inputs a PC and constraints, outputs a new PC satisfying the constraints via convex optimization without retraining, and empirically shows applications like improving performance under scarce data and enforcing fairness measures without compromising fitness.
This work addresses integrating probabilistic propositional logic constraints into the distribution encoded by a probabilistic circuit (PC). PCs are a class of tractable models that allow efficient computations (such as conditional and marginal probabilities) while achieving state-of-the-art performance in some domains. The proposed approach takes both a PC and constraints as inputs, and outputs a new PC that satisfies the constraints. This is done efficiently via convex optimization without the need to retrain the entire model. Empirical evaluations indicate that the combination of constraints and PCs can have multiple use cases, including the improvement of model performance under scarce or incomplete data, as well as the enforcement of machine learning fairness measures into the model without compromising model fitness. We believe that these ideas will open possibilities for multiple other applications involving the combination of logics and deep probabilistic models.