UFAL-CUNI at SemEval-2026 Task 11: An Efficient Modular Neuro-symbolic Method for Syllogistic Reasoning
For researchers in natural language reasoning, this work provides an efficient alternative to large LLMs for syllogistic reasoning, though it is incremental as it combines existing methods.
The authors developed a modular neuro-symbolic system for syllogistic reasoning that combines a symbolic prover with small LLMs (4B parameters), achieving competitive accuracy and low content effect on most subtasks, outperforming LLM-based zero-shot baselines in the same parameter range.
This paper describes our system submitted to SemEval-2026 Task 11: Disentangling Content and Formal Reasoning in Large Language Models. We present an efficient modular neuro-symbolic approach, combining a symbolic prover with small reasoning LLMs (4B parameters). The system consists of an LLM-based parser that translates natural language syllogisms to a first-order logic (FOL) representation, an automated theorem prover, and two optional modules: machine translation for multilingual inputs and a symbolic retrieval component for the identification of relevant premises. The system achieves competitive accuracy and relatively low content effect on most subtasks. Our ablations show that this approach outperforms LLM-based zero-shot baselines in this parameter size range, but also reveal limited multilingual capabilities of small LLMs. Finally, we include a discussion of the task's main ranking metric and analyze its limitations.