EM-NeSy: Expectation Maximization for Neurosymbolic Learning
This work addresses the bottleneck of differentiability in neurosymbolic learning, enabling broader use of approximate inference for robust and interpretable AI.
EM-NeSy casts probabilistic neurosymbolic learning as an instance of the EM algorithm, enabling approximate reasoning without requiring differentiability of the symbolic component, and demonstrates scalability and computational efficiency.
Neurosymbolic (NeSy) models integrate neural networks and symbolic reasoning for robust and interpretable AI. State-of-the-art NeSy models require that the symbolic component is expressed in a differentiable way, often complicating the use of approximate inference. We propose EM-NeSy which casts probabilistic NeSy learning as an instance of the Expectation-Maximization (EM) algorithm. In the expectation step, we compute the posterior over the neurally predicted symbols conditioned on the label via probabilistic inference. In the maximization step, we update the neural parameters based on this posterior using gradient descent only through the neural component. This formulation unlocks the full potential of the EM algorithm for NeSy learning. It allows NeSy to extend naturally to approximate reasoning without any additional modifications or differentiability requirements of the symbolic component. Furthermore, it recovers the standard end-to-end gradient-based NeSy setting under exact inference. Our experimental results demonstrate the scalability and computational efficiency of EM-NeSy.