Shared Global and Local Geometry of Language Model Embeddings
This work provides insights into representation alignment in language models, enabling cross-model transfer techniques, but is incremental in nature.
The paper identifies geometric similarities in token embeddings across large language models, finding both global orientation similarities and local structural patterns, and introduces EMB2EMB to linearly transform steering vectors between models with different dimensions.
Researchers have recently suggested that models share common representations. In our work, we find numerous geometric similarities across the token embeddings of large language models. First, we find ``global'' similarities: token embeddings often share similar relative orientations. Next, we characterize local geometry in two ways: (1) by using Locally Linear Embeddings, and (2) by defining a simple measure for the intrinsic dimension of each embedding. Both characterizations allow us to find local similarities across token embeddings. Additionally, our intrinsic dimension demonstrates that embeddings lie on a lower dimensional manifold, and that tokens with lower intrinsic dimensions often have semantically coherent clusters, while those with higher intrinsic dimensions do not. Based on our findings, we introduce EMB2EMB, a simple application to linearly transform steering vectors from one language model to another, despite the two models having different dimensions.