RS3DBench: A Comprehensive Benchmark for 3D Spatial Perception in Remote Sensing
This addresses the problem of limited 3D spatial perception tools for researchers and practitioners in remote sensing and geographic AI, though it is incremental as it builds on existing datasets and methods.
The authors tackled the lack of comprehensive depth data in remote sensing by introducing RS3DBench, a benchmark with 54,951 image-depth pairs and textual descriptions, and developed a depth estimation model based on stable diffusion that achieves state-of-the-art performance on this dataset.
In this paper, we introduce a novel benchmark designed to propel the advancement of general-purpose, large-scale 3D vision models for remote sensing imagery. While several datasets have been proposed within the realm of remote sensing, many existing collections either lack comprehensive depth information or fail to establish precise alignment between depth data and remote sensing images. To address this deficiency, we present a visual Benchmark for 3D understanding of Remotely Sensed images, dubbed RS3DBench. This dataset encompasses 54,951 pairs of remote sensing images and pixel-level aligned depth maps, accompanied by corresponding textual descriptions, spanning a broad array of geographical contexts. It serves as a tool for training and assessing 3D visual perception models within remote sensing image spatial understanding tasks. Furthermore, we introduce a remotely sensed depth estimation model derived from stable diffusion, harnessing its multimodal fusion capabilities, thereby delivering state-of-the-art performance on our dataset. Our endeavor seeks to make a profound contribution to the evolution of 3D visual perception models and the advancement of geographic artificial intelligence within the remote sensing domain. The dataset, models and code will be accessed on the https://rs3dbench.github.io.