VLRS-Bench: A Vision-Language Reasoning Benchmark for Remote Sensing
For the remote sensing community, this benchmark fills a gap by focusing on complex reasoning beyond perception, providing a standard evaluation tool.
They introduced VLRS-Bench, the first benchmark dedicated to complex reasoning in remote sensing, comprising 2,000 question-answer pairs across 14 tasks. Experiments showed significant bottlenecks in state-of-the-art MLLMs, highlighting the need for improved multimodal reasoning in this domain.
Recent advancements in Multimodal Large Language Models (MLLMs) have enabled complex reasoning. However, existing remote sensing (RS) benchmarks remain heavily biased toward perception tasks, such as object recognition and scene classification. This limitation hinders the development of MLLMs for cognitively demanding RS applications. To address this, we propose a Vision Language ReaSoning Benchmark (VLRS-Bench), which is the first benchmark exclusively dedicated to complex RS reasoning. Structured across the three core dimensions of Cognition, Decision, and Prediction, VLRS-Bench comprises 2,000 question-answer pairs with an average question length of 130.19 words, spanning 14 tasks and up to eight temporal phases. VLRS-Bench is constructed via a specialized pipeline that integrates RS-specific priors and expert knowledge to ensure geospatial realism and reasoning complexity. Experimental results reveal significant bottlenecks in existing state-of-the-art MLLMs, providing critical insights for advancing multimodal reasoning within the remote sensing community. The project repository is available at https://github.com/MiliLab/VLRS-Bench.