Topic Aware Probing: From Sentence Length Prediction to Idiom Identification how reliant are Neural Language Models on Topic?
This work addresses a fundamental debate in NLP about the reliance of state-of-the-art models on topic versus syntactic information, with implications for model interpretability and design, though it is incremental in probing methodology.
The study investigated whether Transformer-based language models like BERT and RoBERTa rely on topic information for processing language, using a novel topic-aware probing method across tasks from sentence length prediction to idiom identification. Results showed that these models encode both topic and non-topic information, with idiom identification primarily dependent on topic encoding, and tasks insensitive to topic were more difficult for the models.
Transformer-based Neural Language Models achieve state-of-the-art performance on various natural language processing tasks. However, an open question is the extent to which these models rely on word-order/syntactic or word co-occurrence/topic-based information when processing natural language. This work contributes to this debate by addressing the question of whether these models primarily use topic as a signal, by exploring the relationship between Transformer-based models' (BERT and RoBERTa's) performance on a range of probing tasks in English, from simple lexical tasks such as sentence length prediction to complex semantic tasks such as idiom token identification, and the sensitivity of these tasks to the topic information. To this end, we propose a novel probing method which we call topic-aware probing. Our initial results indicate that Transformer-based models encode both topic and non-topic information in their intermediate layers, but also that the facility of these models to distinguish idiomatic usage is primarily based on their ability to identify and encode topic. Furthermore, our analysis of these models' performance on other standard probing tasks suggests that tasks that are relatively insensitive to the topic information are also tasks that are relatively difficult for these models.