Layers at Similar Depths Generate Similar Activations Across LLM Architectures
This work addresses the problem of understanding latent space similarities across LLMs for researchers and practitioners, providing insights into model interpretability and alignment, but it is incremental as it builds on existing studies of neural network representations.
The study investigated how latent spaces in independently-trained large language models (LLMs) relate by analyzing nearest neighbor relationships of activations across layers in 24 open-weight LLMs, finding that these relationships vary within models but are shared between corresponding layers of different models, indicating a common progression of activation geometries adapted to different architectures.
How do the latent spaces used by independently-trained LLMs relate to one another? We study the nearest neighbor relationships induced by activations at different layers of 24 open-weight LLMs, and find that they 1) tend to vary from layer to layer within a model, and 2) are approximately shared between corresponding layers of different models. Claim 2 shows that these nearest neighbor relationships are not arbitrary, as they are shared across models, but Claim 1 shows that they are not "obvious" either, as there is no single set of nearest neighbor relationships that is universally shared. Together, these suggest that LLMs generate a progression of activation geometries from layer to layer, but that this entire progression is largely shared between models, stretched and squeezed to fit into different architectures.