Emergence of a High-Dimensional Abstraction Phase in Language Transformers
This work provides insights into the internal workings of language models, which could help improve model design and interpretability for natural language processing researchers.
The study analyzed the geometric properties of language models and identified a high-dimensional abstraction phase that correlates with the first full linguistic abstraction of input, viable transfer to downstream tasks, and better language modeling performance across five transformer-based models and three datasets.
A language model (LM) is a mapping from a linguistic context to an output token. However, much remains to be known about this mapping, including how its geometric properties relate to its function. We take a high-level geometric approach to its analysis, observing, across five pre-trained transformer-based LMs and three input datasets, a distinct phase characterized by high intrinsic dimensionality. During this phase, representations (1) correspond to the first full linguistic abstraction of the input; (2) are the first to viably transfer to downstream tasks; (3) predict each other across different LMs. Moreover, we find that an earlier onset of the phase strongly predicts better language modelling performance. In short, our results suggest that a central high-dimensionality phase underlies core linguistic processing in many common LM architectures.