Enhancing Systematic Decompositional Natural Language Inference Using Informal Logic
This work addresses the challenge of determining valid compositional entailment for researchers in natural language processing, though it is incremental as it builds on existing neuro-symbolic methods.
The paper tackled the problem of noisy datasets and limited performance in decompositional natural language inference by introducing a consistent annotation approach, resulting in a new dataset (RDTE) with 9% higher internal consistency and improved accuracy and proof quality in entailment tree reasoning.
Recent language models enable new opportunities for structured reasoning with text, such as the construction of intuitive, proof-like textual entailment trees without relying on brittle formal logic. However, progress in this direction has been hampered by a long-standing lack of a clear protocol for determining what valid compositional entailment is. This absence causes noisy datasets and limited performance gains by modern neuro-symbolic engines. To address these problems, we formulate a consistent and theoretically grounded approach to annotating decompositional entailment and evaluate its impact on LLM-based textual inference. We find that our new dataset, RDTE (Recognizing Decompositional Textual Entailment), has a substantially higher internal consistency (+9%) than prior decompositional entailment datasets. We also find that training an RDTE-oriented entailment classifier via knowledge distillation and employing it in an entailment tree reasoning engine significantly improves both accuracy and proof quality, illustrating the practical benefit of this advance for textual inference.