Developmental Negation Processing in Transformer Language Models
This work addresses the challenge of negation reasoning in language models for NLP researchers, but it is incremental as it focuses on specific categories without major breakthroughs.
The study investigated how well transformer language models process categories of negation studied in developmental psychology by framing it as a natural language inference task, finding that models perform consistently better only on certain categories, indicating clear distinctions in processing.
Reasoning using negation is known to be difficult for transformer-based language models. While previous studies have used the tools of psycholinguistics to probe a transformer's ability to reason over negation, none have focused on the types of negation studied in developmental psychology. We explore how well transformers can process such categories of negation, by framing the problem as a natural language inference (NLI) task. We curate a set of diagnostic questions for our target categories from popular NLI datasets and evaluate how well a suite of models reason over them. We find that models perform consistently better only on certain categories, suggesting clear distinctions in how they are processed.