Lexicosyntactic Inference in Neural Models
This work addresses the problem of understanding neural models' limitations in linguistic inference for NLP researchers, but it is incremental as it focuses on a specific dataset and error analysis.
The authors investigated neural models' ability to capture lexicosyntactic inferences, using event factuality prediction as a case study, and found that state-of-the-art systems make systematic errors in this task.
We investigate neural models' ability to capture lexicosyntactic inferences: inferences triggered by the interaction of lexical and syntactic information. We take the task of event factuality prediction as a case study and build a factuality judgment dataset for all English clause-embedding verbs in various syntactic contexts. We use this dataset, which we make publicly available, to probe the behavior of current state-of-the-art neural systems, showing that these systems make certain systematic errors that are clearly visible through the lens of factuality prediction.