Constraints-Guided Diffusion Reasoner for Neuro-Symbolic Learning
This work addresses the problem of neuro-symbolic learning for AI systems, but it appears incremental as it builds on existing diffusion models and optimization techniques.
The paper tackled the challenge of enabling neural networks to learn complex logical constraints for symbolic reasoning by using a diffusion-based pipeline with a two-stage training strategy and fine-tuning via an improved proximal policy optimization algorithm, achieving outstanding accuracy and logical consistency on benchmarks like Sudoku and Maze.
Enabling neural networks to learn complex logical constraints and fulfill symbolic reasoning is a critical challenge. Bridging this gap often requires guiding the neural network's output distribution to move closer to the symbolic constraints. While diffusion models have shown remarkable generative capability across various domains, we employ the powerful architecture to perform neuro-symbolic learning and solve logical puzzles. Our diffusion-based pipeline adopts a two-stage training strategy: the first stage focuses on cultivating basic reasoning abilities, while the second emphasizes systematic learning of logical constraints. To impose hard constraints on neural outputs in the second stage, we formulate the diffusion reasoner as a Markov decision process and innovatively fine-tune it with an improved proximal policy optimization algorithm. We utilize a rule-based reward signal derived from the logical consistency of neural outputs and adopt a flexible strategy to optimize the diffusion reasoner's policy. We evaluate our methodology on some classical symbolic reasoning benchmarks, including Sudoku, Maze, pathfinding and preference learning. Experimental results demonstrate that our approach achieves outstanding accuracy and logical consistency among neural networks.