Geometry of Ordinal Representations in Language Models
For researchers studying mechanistic interpretability of language models, this paper provides a nuanced understanding of when and how ordinal representations form geometric structures, revealing limitations of the 1D manifold hypothesis.
This work investigates whether language models represent ordinal variables (e.g., bracket depth, numeric magnitude) on curved 1D manifolds, finding that such geometry emerges only when the ordinal variable is locally computable from token identity; tasks requiring cross-position integration yield higher-dimensional representations. The study also reveals architecture-dependent differences in geometric computation, with Qwen3-4B showing stronger twisting than Gemma models for indentation.
Recent work showed that language models represent character counts on curved 1D manifolds, with attention heads performing geometric transformations to enable computation. We test whether this generalizes across four ordinal tasks (bracket depth, indentation, table position, numeric magnitude) in Gemma-2-2B, Gemma-2-9B, and Qwen3-4B. We find that 1D manifolds with place-cell feature tiling emerge for tasks where the ordinal variable is locally computable from token identity, while tasks requiring cross-position integration or semantic extraction produce higher-dimensional or incoherent representations. Geometric computation is architecture-dependent: Qwen3-4B shows substantially stronger twisting than Gemma models for indentation, and its twisters preserve ordinal order, unlike its numeric twisters. Activation patching confirms that the identified manifold subspaces concentrate task-relevant information, with manifold-direction ablation causing dramatically larger probe accuracy drops than random-direction controls.