Machine Reading, Fast and Slow: When Do Models "Understand" Language?
This addresses the challenge of model interpretability and generalization in NLU for researchers, but it is incremental as it builds on existing methods to analyze model behavior without introducing new techniques.
The paper tackled the problem of whether deep learning models in natural language understanding rely on correct linguistic reasoning, specifically for coreference resolution and comparison tasks, finding that larger BERT-based models show some improvement in using appropriate information for comparison but still fail to generalize, indicating reliance on lexical patterns rather than principles.
Two of the most fundamental challenges in Natural Language Understanding (NLU) at present are: (a) how to establish whether deep learning-based models score highly on NLU benchmarks for the 'right' reasons; and (b) to understand what those reasons would even be. We investigate the behavior of reading comprehension models with respect to two linguistic 'skills': coreference resolution and comparison. We propose a definition for the reasoning steps expected from a system that would be 'reading slowly', and compare that with the behavior of five models of the BERT family of various sizes, observed through saliency scores and counterfactual explanations. We find that for comparison (but not coreference) the systems based on larger encoders are more likely to rely on the 'right' information, but even they struggle with generalization, suggesting that they still learn specific lexical patterns rather than the general principles of comparison.