When Does Meaning Backfire? Investigating the Role of AMRs in NLI
This work addresses the problem of semantic parsing in NLI for researchers, showing incremental insights into how AMR integration can backfire.
The study investigated whether adding Abstract Meaning Representation (AMR) semantic information improves pretrained language models in Natural Language Inference (NLI), finding that fine-tuning with AMR hinders generalization while prompting with AMR in GPT-4o yields slight gains, but these gains stem from amplifying surface-level differences rather than aiding semantic reasoning.
Natural Language Inference (NLI) relies heavily on adequately parsing the semantic content of the premise and hypothesis. In this work, we investigate whether adding semantic information in the form of an Abstract Meaning Representation (AMR) helps pretrained language models better generalize in NLI. Our experiments integrating AMR into NLI in both fine-tuning and prompting settings show that the presence of AMR in fine-tuning hinders model generalization while prompting with AMR leads to slight gains in GPT-4o. However, an ablation study reveals that the improvement comes from amplifying surface-level differences rather than aiding semantic reasoning. This amplification can mislead models to predict non-entailment even when the core meaning is preserved.