From Human Cognition to Neural Activations: Probing the Computational Primitives of Spatial Reasoning in LLMs
This work addresses the problem of understanding spatial intelligence in LLMs for AI researchers, revealing mechanistic limitations that benchmark accuracy alone cannot capture, though it is incremental in building on existing cognitive theories.
The study investigated whether large language models (LLMs) have structured internal spatial representations or rely on linguistic heuristics by decomposing spatial reasoning into primitives and analyzing models in multiple languages. It found that spatial information is encoded transiently and fragmented, with limited integration into predictions, indicating context-dependent rather than robust spatial reasoning.
As spatial intelligence becomes an increasingly important capability for foundation models, it remains unclear whether large language models' (LLMs) performance on spatial reasoning benchmarks reflects structured internal spatial representations or reliance on linguistic heuristics. We address this question from a mechanistic perspective by examining how spatial information is internally represented and used. Drawing on computational theories of human spatial cognition, we decompose spatial reasoning into three primitives, relational composition, representational transformation, and stateful spatial updating, and design controlled task families for each. We evaluate multilingual LLMs in English, Chinese, and Arabic under single pass inference, and analyze internal representations using linear probing, sparse autoencoder based feature analysis, and causal interventions. We find that task relevant spatial information is encoded in intermediate layers and can causally influence behavior, but these representations are transient, fragmented across task families, and weakly integrated into final predictions. Cross linguistic analysis further reveals mechanistic degeneracy, where similar behavioral performance arises from distinct internal pathways. Overall, our results suggest that current LLMs exhibit limited and context dependent spatial representations rather than robust, general purpose spatial reasoning, highlighting the need for mechanistic evaluation beyond benchmark accuracy.