EmbSpatial-Bench: Benchmarking Spatial Understanding for Embodied Tasks with Large Vision-Language Models
This addresses the gap in assessing spatial skills for embodied intelligence, which is crucial for robotics and AI applications, but is incremental as it builds on existing benchmarking efforts.
The authors tackled the problem of evaluating spatial understanding in Large Vision-Language Models (LVLMs) for embodied tasks by constructing EmbSpatial-Bench, a benchmark covering 6 spatial relationships, and found that current LVLMs, including GPT-4V, have insufficient capacity in this area.
The recent rapid development of Large Vision-Language Models (LVLMs) has indicated their potential for embodied tasks.However, the critical skill of spatial understanding in embodied environments has not been thoroughly evaluated, leaving the gap between current LVLMs and qualified embodied intelligence unknown. Therefore, we construct EmbSpatial-Bench, a benchmark for evaluating embodied spatial understanding of LVLMs.The benchmark is automatically derived from embodied scenes and covers 6 spatial relationships from an egocentric perspective.Experiments expose the insufficient capacity of current LVLMs (even GPT-4V). We further present EmbSpatial-SFT, an instruction-tuning dataset designed to improve LVLMs' embodied spatial understanding.