MonoSR: Open-Vocabulary Spatial Reasoning from Monocular Images
This work addresses the need for open-vocabulary spatial reasoning in real-world applications like embodied AI and autonomous driving, but it is incremental as it focuses on dataset creation and analysis rather than a novel method.
The paper tackles the problem of spatial reasoning from monocular images, which is limited in existing research, by introducing MonoSR, a large-scale dataset spanning indoor, outdoor, and object-centric scenarios with multiple question types, and evaluates advanced vision-language models to reveal their limitations and provide guidance for future models.
Spatial reasoning (SR), the ability to infer 3D spatial information from 2D inputs, is essential for real-world applications such as embodied AI and autonomous driving. However, existing research primarily focuses on indoor environments and typically relies on multi-view observations, which limits their generalizability to outdoor scenarios and constrains their applicability to monocular images, the most common real-world setting. In this work, we propose MonoSR, a large-scale monocular spatial reasoning dataset that spans diverse scenarios including indoor, outdoor, and object-centric settings, and supports multiple question types. MonoSR provides a path toward open-world monocular spatial reasoning. Beyond introducing the dataset, we evaluate advanced vision-language models to reveal their limitations on this challenging task. We further analyze whether auxiliary information is crucial for monocular spatial reasoning and offer practical guidance for designing future models. These contributions collectively establish a foundation for advancing monocular spatial reasoning in real-world, open-world environments.