Shape Happens: Automatic Feature Manifold Discovery in LLMs via Supervised Multi-Dimensional Scaling
This provides insights into the functional role of feature manifolds for understanding entity-based reasoning in language models, though it is incremental as it builds on the linear representation hypothesis.
The paper tackled the problem of automatically discovering feature manifolds in language models, introducing Supervised Multi-Dimensional Scaling (SMDS) and applying it to temporal reasoning, revealing geometric structures like circles and lines that reflect concept properties and support reasoning.
The linear representation hypothesis states that language models (LMs) encode concepts as directions in their latent space, forming organized, multidimensional manifolds. Prior efforts focus on discovering specific geometries for specific features, and thus lack generalization. We introduce Supervised Multi-Dimensional Scaling (SMDS), a model-agnostic method to automatically discover feature manifolds. We apply SMDS to temporal reasoning as a case study, finding that different features form various geometric structures such as circles, lines, and clusters. SMDS reveals many insights on these structures: they consistently reflect the properties of the concepts they represent; are stable across model families and sizes; actively support reasoning in models; and dynamically reshape in response to context changes. Together, our findings shed light on the functional role of feature manifolds, supporting a model of entity-based reasoning in which LMs encode and transform structured representations.