PA-SAM: Prompt Adapter SAM for High-Quality Image Segmentation
This work addresses segmentation quality issues in SAM for users in computer vision applications, representing an incremental improvement.
The paper tackles the problem of improving mask prediction quality in the Segment Anything Model (SAM) for image segmentation, especially in real-world scenarios, by introducing a prompt-driven adapter (PA-SAM) that enhances segmentation performance, outperforming other SAM-based methods in high-quality, zero-shot, and open-set segmentation.
The Segment Anything Model (SAM) has exhibited outstanding performance in various image segmentation tasks. Despite being trained with over a billion masks, SAM faces challenges in mask prediction quality in numerous scenarios, especially in real-world contexts. In this paper, we introduce a novel prompt-driven adapter into SAM, namely Prompt Adapter Segment Anything Model (PA-SAM), aiming to enhance the segmentation mask quality of the original SAM. By exclusively training the prompt adapter, PA-SAM extracts detailed information from images and optimizes the mask decoder feature at both sparse and dense prompt levels, improving the segmentation performance of SAM to produce high-quality masks. Experimental results demonstrate that our PA-SAM outperforms other SAM-based methods in high-quality, zero-shot, and open-set segmentation. We're making the source code and models available at https://github.com/xzz2/pa-sam.