A Mathematical Framework and a Suite of Learning Techniques for Neural-Symbolic Systems
This work addresses the need for a cohesive framework to organize and advance NeSy modeling, which is incremental in providing a structured approach for researchers and practitioners in AI.
The authors tackled the lack of a unifying framework for Neural-Symbolic (NeSy) systems by introducing Neural-Symbolic Energy-Based Models (NeSy-EBMs), which provide a mathematical foundation and enable general gradient expressions and learning approaches, demonstrated through empirical analysis across multiple datasets.
The field of Neural-Symbolic (NeSy) systems is growing rapidly. Proposed approaches show great promise in achieving symbiotic unions of neural and symbolic methods. However, a unifying framework is needed to organize common NeSy modeling patterns and develop general learning approaches. In this paper, we introduce Neural-Symbolic Energy-Based Models (NeSy-EBMs), a unifying mathematical framework for discriminative and generative NeSy modeling. Importantly, NeSy-EBMs allow the derivation of general expressions for gradients of prominent learning losses, and we introduce a suite of four learning approaches that leverage methods from multiple domains, including bilevel and stochastic policy optimization. Finally, we ground the NeSy-EBM framework with Neural Probabilistic Soft Logic (NeuPSL), an open-source NeSy-EBM library designed for scalability and expressivity, facilitating the real-world application of NeSy systems. Through extensive empirical analysis across multiple datasets, we demonstrate the practical advantages of NeSy-EBMs in various tasks, including image classification, graph node labeling, autonomous vehicle situation awareness, and question answering.