Do Language Models Understand Anything? On the Ability of LSTMs to Understand Negative Polarity Items
This work addresses the interpretability of language models for linguists and AI researchers, but it is incremental as it focuses on a specific phenomenon without broad performance gains.
The paper tackled the problem of whether neural language models, specifically LSTMs, can understand complex linguistic phenomena like negative polarity items, and found that the model correctly processes these constructions by relating licensing contexts to the items and being aware of their scope.
In this paper, we attempt to link the inner workings of a neural language model to linguistic theory, focusing on a complex phenomenon well discussed in formal linguis- tics: (negative) polarity items. We briefly discuss the leading hypotheses about the licensing contexts that allow negative polarity items and evaluate to what extent a neural language model has the ability to correctly process a subset of such constructions. We show that the model finds a relation between the licensing context and the negative polarity item and appears to be aware of the scope of this context, which we extract from a parse tree of the sentence. With this research, we hope to pave the way for other studies linking formal linguistics to deep learning.