CVJul 30

ViewMind3D: Modular View-Aware Inference for Training-Free 3D-QA

arXiv:2607.284427.0
Predicted impact top 48% in CV · last 90 daysOriginality Incremental advance
AI Analysis

For embodied AI and robotic perception, this provides a scalable, annotation-free alternative to 3D-specific training, though gains are specific to certain question types.

ViewMind3D is a training-free modular framework for 3D question answering that decomposes the task into four interpretable components, achieving competitive performance on ScanQA and SQA3D, with 50.8% accuracy on SQA3D and 73.4 CIDEr on ScanQA, particularly improving spatially grounded question types.

Recent advances in large language models (LLMs) and vision-language models (VLMs) have enabled new possibilities for 3D question answering (3D-QA), a key capability for embodied AI and robotic perception. However, most existing methods rely on 3D-specific training or fine-tuning with costly annotations, limiting their scalability and real-world applicability. We present \textbf{ViewMind3D}, a fully training-free and modular framework for 3D spatial reasoning over multi-view observations of a scene without requiring complete 3D reconstruction. The framework decomposes the 3D-QA task into four interpretable components: (1) question-driven multi-view selection, (2) guided visual grounding with language-conditioned object cues, (3) spatial context encoding via a bird's-eye-view (BEV) viewpoint indicator, and (4) structured answer generation through role-based reasoning. This design enables structured, robust, and interpretable reasoning without requiring model tuning. Experimental results on ScanQA and SQA3D show that ViewMind3D achieves competitive performance compared to prior training-free and fine-tuned 3D-LLMs. In particular, our method improves performance on spatially grounded question types, such as ``What'' questions in SQA3D, while maintaining strong overall accuracy (50.8\%) and achieving 73.4 CIDEr on ScanQA. These results demonstrate that effective 3D reasoning can be achieved through modular orchestration of general-purpose LLMs and VLMs for robotic perception in real-world environments.

Foundations

The foundational work for this paper's niche, ranked by how specifically the neighbourhood builds on it — not by global fame.

Your Notes