CVJan 19

Think3D: Thinking with Space for Spatial Reasoning

arXiv:2601.13029v19 citationsHas Code
Originality Incremental advance
AI Analysis

This addresses the limitation of VLMs in 3D reasoning for AI agents, offering a training-free tool-augmented approach that is incremental but establishes a new dimension in multimodal intelligence.

The paper tackles the problem of spatial reasoning in vision large models (VLMs), which struggle with 3D understanding, by introducing Think3D, a framework that enables VLMs to interact with 3D reconstructions, resulting in average gains of +7.8% on BLINK Multi-view and MindCube and +4.7% on VSI-Bench for models like GPT-4.1 and Gemini 2.5 Pro.

Understanding and reasoning about the physical world requires spatial intelligence: the ability to interpret geometry, perspective, and spatial relations beyond 2D perception. While recent vision large models (VLMs) excel at visual understanding, they remain fundamentally 2D perceivers and struggle with genuine 3D reasoning. We introduce Think3D, a framework that enables VLM agents to think with 3D space. By leveraging 3D reconstruction models that recover point clouds and camera poses from images or videos, Think3D allows the agent to actively manipulate space through camera-based operations and ego/global-view switching, transforming spatial reasoning into an interactive 3D chain-of-thought process. Without additional training, Think3D significantly improves the spatial reasoning performance of advanced models such as GPT-4.1 and Gemini 2.5 Pro, yielding average gains of +7.8% on BLINK Multi-view and MindCube, and +4.7% on VSI-Bench. We further show that smaller models, which struggle with spatial exploration, benefit significantly from a reinforcement learning policy that enables the model to select informative viewpoints and operations. With RL, the benefit from tool usage increases from +0.7% to +6.8%. Our findings demonstrate that training-free, tool-augmented spatial exploration is a viable path toward more flexible and human-like 3D reasoning in multimodal agents, establishing a new dimension of multimodal intelligence. Code and weights are released at https://github.com/zhangzaibin/spagent.

Foundations

The foundational work for this paper's niche, ranked by how specifically the neighbourhood builds on it — not by global fame.

Your Notes