NeuralFastLAS: Fast Logic-Based Learning from Raw Data
This addresses the scalability issue for researchers and practitioners in neuro-symbolic AI, offering a faster and more stable end-to-end approach, though it is incremental as it builds on existing neuro-symbolic frameworks.
The paper tackles the problem of slow and unstable joint training in neuro-symbolic learning by introducing NeuralFastLAS, which computes a relevant set of rules, trains a neural network with a posterior distribution on rules for stability, and finds an optimal symbolic solution. It achieves state-of-the-art accuracy in arithmetic and logical tasks with training times up to 100 times faster than other methods.
Symbolic rule learners generate interpretable solutions, however they require the input to be encoded symbolically. Neuro-symbolic approaches overcome this issue by mapping raw data to latent symbolic concepts using a neural network. Training the neural and symbolic components jointly is difficult, due to slow and unstable learning, hence many existing systems rely on hand-engineered rules to train the network. We introduce NeuralFastLAS, a scalable and fast end-to-end approach that trains a neural network jointly with a symbolic learner. For a given task, NeuralFastLAS computes a relevant set of rules, proved to contain an optimal symbolic solution, trains a neural network using these rules, and finally finds an optimal symbolic solution to the task while taking network predictions into account. A key novelty of our approach is learning a posterior distribution on rules while training the neural network to improve stability during training. We provide theoretical results for a sufficient condition on network training to guarantee correctness of the final solution. Experimental results demonstrate that NeuralFastLAS is able to achieve state-of-the-art accuracy in arithmetic and logical tasks, with a training time that is up to two orders of magnitude faster than other jointly trained neuro-symbolic methods.