EXPLAIN, AGREE, LEARN: Scaling Learning for Neural Probabilistic Logic
This work addresses scalability challenges for researchers and practitioners in neuro-symbolic AI, though it is incremental as it builds on existing neural probabilistic logic frameworks.
The authors tackled the scalability issue in neural probabilistic logic systems by proposing a sampling-based objective to avoid expensive probabilistic inference, and demonstrated that their EXAL method outperforms recent neuro-symbolic methods on scaled-up MNIST addition and Warcraft pathfinding problems.
Neural probabilistic logic systems follow the neuro-symbolic (NeSy) paradigm by combining the perceptive and learning capabilities of neural networks with the robustness of probabilistic logic. Learning corresponds to likelihood optimization of the neural networks. However, to obtain the likelihood exactly, expensive probabilistic logic inference is required. To scale learning to more complex systems, we therefore propose to instead optimize a sampling based objective. We prove that the objective has a bounded error with respect to the likelihood, which vanishes when increasing the sample count. Furthermore, the error vanishes faster by exploiting a new concept of sample diversity. We then develop the EXPLAIN, AGREE, LEARN (EXAL) method that uses this objective. EXPLAIN samples explanations for the data. AGREE reweighs each explanation in concordance with the neural component. LEARN uses the reweighed explanations as a signal for learning. In contrast to previous NeSy methods, EXAL can scale to larger problem sizes while retaining theoretical guarantees on the error. Experimentally, our theoretical claims are verified and EXAL outperforms recent NeSy methods when scaling up the MNIST addition and Warcraft pathfinding problems.