Symbol-Equivariant Recurrent Reasoning Models
This work addresses the problem of improving neural network robustness and scalability for structured reasoning tasks like puzzles and AGI benchmarks, representing an incremental advance by explicitly encoding symmetries in existing architectures.
The paper tackled the challenge of solving reasoning problems like Sudoku and ARC-AGI with neural networks by introducing Symbol-Equivariant Recurrent Reasoning Models (SE-RRMs), which enforce permutation equivariance to handle symbol symmetries explicitly, resulting in outperforming prior models on 9x9 Sudoku and generalizing to larger instances without additional training.
Reasoning problems such as Sudoku and ARC-AGI remain challenging for neural networks. The structured problem solving architecture family of Recurrent Reasoning Models (RRMs), including Hierarchical Reasoning Model (HRM) and Tiny Recursive Model (TRM), offer a compact alternative to large language models, but currently handle symbol symmetries only implicitly via costly data augmentation. We introduce Symbol-Equivariant Recurrent Reasoning Models (SE-RRMs), which enforce permutation equivariance at the architectural level through symbol-equivariant layers, guaranteeing identical solutions under symbol or color permutations. SE-RRMs outperform prior RRMs on 9x9 Sudoku and generalize from just training on 9x9 to smaller 4x4 and larger 16x16 and 25x25 instances, to which existing RRMs cannot extrapolate. On ARC-AGI-1 and ARC-AGI-2, SE-RRMs achieve competitive performance with substantially less data augmentation and only 2 million parameters, demonstrating that explicitly encoding symmetry improves the robustness and scalability of neural reasoning. Code is available at https://github.com/ml-jku/SE-RRM.