RSPrompter: Learning to Prompt for Remote Sensing Instance Segmentation based on Visual Foundation Model
This work addresses the need for automated, category-aware segmentation in remote sensing, which is incremental as it builds on the existing SAM foundation.
The authors tackled the problem of automating instance segmentation for remote sensing images by adapting the Segment Anything Model (SAM) with semantic prompts, achieving effective results validated on datasets like WHU building, NWPU VHR-10, and SSDD.
Leveraging the extensive training data from SA-1B, the Segment Anything Model (SAM) demonstrates remarkable generalization and zero-shot capabilities. However, as a category-agnostic instance segmentation method, SAM heavily relies on prior manual guidance, including points, boxes, and coarse-grained masks. Furthermore, its performance in remote sensing image segmentation tasks remains largely unexplored and unproven. In this paper, we aim to develop an automated instance segmentation approach for remote sensing images, based on the foundational SAM model and incorporating semantic category information. Drawing inspiration from prompt learning, we propose a method to learn the generation of appropriate prompts for SAM. This enables SAM to produce semantically discernible segmentation results for remote sensing images, a concept we have termed RSPrompter. We also propose several ongoing derivatives for instance segmentation tasks, drawing on recent advancements within the SAM community, and compare their performance with RSPrompter. Extensive experimental results, derived from the WHU building, NWPU VHR-10, and SSDD datasets, validate the effectiveness of our proposed method. The code for our method is publicly available at kychen.me/RSPrompter.