Revisiting Anisotropy in Language Transformers: The Geometry of Learning Dynamics
For NLP researchers, it offers a geometric understanding of anisotropy in Transformers, but the contribution is incremental as it extends existing theoretical work.
The paper derives geometric arguments for anisotropy in Transformer language models and provides empirical evidence that activation-derived tangent directions capture most gradient anisotropy, supporting a tangent-aligned explanation.
Since their introduction, Transformer architectures have dominated Natural Language Processing (NLP). However, recent research has highlighted an inherent anisotropy phenomenon in these models, presenting a significant challenge to their geometric interpretation. Previous theoretical studies on this phenomenon are rarely grounded in the underlying representation geometry. In this paper, we extend them by deriving geometric arguments for how frequency-biased sampling attenuates curvature visibility and why training preferentially amplify tangent directions. Empirically, we then use concept-based mechanistic interpretability during training, rather than only post hoc, to fit activation-derived low-rank tangent proxies and test them against ordinary backpropagated true gradients. Across encoder-style and decoder-style language models, we find that these activation-derived directions capture both unusually large gradient energy and a substantially larger share of gradient anisotropy than matched-rank normal controls, providing strong empirical support for a tangent-aligned account of anisotropy.