SpatialReasoner: Towards Explicit and Generalizable 3D Spatial Reasoning
This addresses the challenge of 3D spatial reasoning for AI systems, particularly in multi-modal models, by making reasoning more explicit and generalizable, though it builds incrementally on existing approaches.
The paper tackles the problem of 3D spatial reasoning in vision-language models by introducing SpatialReasoner, which uses explicit 3D representations across perception, computation, and reasoning stages. The result shows improved performance, outperforming Gemini 2.0 by 9.2% on 3DSRBench and better generalization to novel question types.
Despite recent advances on multi-modal models, 3D spatial reasoning remains a challenging task for state-of-the-art open-source and proprietary models. Recent studies explore data-driven approaches and achieve enhanced spatial reasoning performance by fine-tuning models on 3D-related visual question-answering data. However, these methods typically perform spatial reasoning in an implicit manner and often fail on questions that are trivial to humans, even with long chain-of-thought reasoning. In this work, we introduce SpatialReasoner, a novel large vision-language model (LVLM) that addresses 3D spatial reasoning with explicit 3D representations shared between multiple stages--3D perception, computation, and reasoning. Explicit 3D representations provide a coherent interface that supports advanced 3D spatial reasoning and improves the generalization ability to novel question types. Furthermore, by analyzing the explicit 3D representations in multi-step reasoning traces of SpatialReasoner, we study the factual errors and identify key shortcomings of current LVLMs. Results show that our SpatialReasoner achieves improved performance on a variety of spatial reasoning benchmarks, outperforming Gemini 2.0 by 9.2% on 3DSRBench, and generalizes better when evaluating on novel 3D spatial reasoning questions. Our study bridges the 3D parsing capabilities of prior visual foundation models with the powerful reasoning abilities of large language models, opening new directions for 3D spatial reasoning.