Lattice Deduction Transformers
For AI reasoning tasks, LDT provides a highly efficient and accurate neuro-symbolic approach that outperforms large language models on structured puzzles.
The Lattice Deduction Transformer (LDT) achieves 100% accuracy on Sudoku-Extreme and Snowflake Sudoku, and 99.9% on Maze-Hard with far fewer parameters than prior models, while frontier LLMs score 0%.
We introduce the Lattice Deduction Transformer (LDT), a recurrent transformer that approximates logically sound deduction by projecting its latent state through a lattice between forward passes. We train on-policy in a process that mirrors deduction in a search-based constraint solver and supervise training via a domain-agnostic, abstract-interpretation-based approximation of the set of solution candidates. An $800$K-parameter LDT achieves $100\%$ accuracy on Sudoku-Extreme and Snowflake Sudoku, at a fraction of the training cost of prior small recurrent reasoners, while remaining empirically sound: the model returns a correct answer or abstains. A $1.8$M-parameter variant reaches $99.9\%$ accuracy on Maze-Hard. Frontier LLMs score $0\%$ on all three benchmarks.