Detecting Conceptual Abstraction in LLMs
This work addresses the explainability of conceptual abstraction in LLMs, which is an incremental step for researchers in natural language processing and AI interpretability.
The researchers tackled the problem of detecting noun abstraction in large language models by analyzing attention matrices from BERT using psychologically motivated noun pairs and surface patterns for hypernymy, showing that hypernymy detection is not solely based on distributional similarity.
We present a novel approach to detecting noun abstraction within a large language model (LLM). Starting from a psychologically motivated set of noun pairs in taxonomic relationships, we instantiate surface patterns indicating hypernymy and analyze the attention matrices produced by BERT. We compare the results to two sets of counterfactuals and show that we can detect hypernymy in the abstraction mechanism, which cannot solely be related to the distributional similarity of noun pairs. Our findings are a first step towards the explainability of conceptual abstraction in LLMs.