SPHERE: Unveiling Spatial Blind Spots in Vision-Language Models Through Hierarchical Evaluation
This work addresses a critical gap in vision-language models for applications requiring human-like spatial understanding, though it is incremental as it focuses on evaluation rather than new model development.
The paper tackled the problem of limited spatial reasoning in vision-language models by developing SPHERE, a hierarchical evaluation framework and dataset, which revealed significant deficiencies in models, such as struggles with distance, perspective, and spatial logic.
Current vision-language models may grasp basic spatial cues and simple directions (e.g. left, right, front, back), but struggle with the multi-dimensional spatial reasoning necessary for human-like understanding and real-world applications. To address this gap, we develop SPHERE (Spatial Perception and Hierarchical Evaluation of REasoning), a hierarchical evaluation framework supported by a new human-annotated dataset. SPHERE systematically probes models across increasing levels of complexity, from fundamental skills to multi-skill integration and high-level reasoning that combines spatial, visual, and logical understanding. Benchmark evaluation of state-of-the-art models reveals significant deficiencies, especially in reasoning about distance and proximity, understanding both egocentric and allocentric perspectives, and applying spatial logic in physical contexts. These findings expose critical blind spots in existing models and underscore the need for more advanced spatial reasoning techniques, driving the development of vision-language models that align more closely with human spatial cognition. The SPHERE benchmark is available at https://github.com/zwenyu/SPHERE-VLM.