Semantic Structure of Feature Space in Large Language Models
For researchers in AI interpretability and cognitive science, this work provides evidence that LLMs encode semantic structure similar to human cognition, but the findings are incremental as they extend known alignment results.
The paper shows that geometric relations between semantic features in LLM hidden states mirror human psychological associations, with high correlations between model projections and human ratings on 32 semantic axes, and that steering effects on one axis spill over proportionally to cosine similarities between axes.
We show that the geometric relations between semantic features in large language models' hidden states closely mirror human psychological associations. We construct feature vectors corresponding to 360 words and project them on 32 semantic axes (e.g. beautiful-ugly, soft-hard), and find that these projections correlate highly with human ratings of those words on the respective semantic scales. Second, we find that the cosine similarities between the semantic axes themselves are highly predictive of the correlations between these scales in the survey. Third, we show that substantial variance across the 32 semantic axes lies on a low-dimensional subspace, reproducing patterns typical of human semantic associations. Finally, we demonstrate that steering a word on one semantic axis causes spillover effects on the model's rating of that word on other semantic scales proportionate to the cosine similarity between those semantic axes. These findings suggest that features should be understood not only in isolation but through their geometric relations and the meaningful subspaces they form.