SAM3-UNet: Simplified Adaptation of Segment Anything Model 3
This work addresses the need for efficient adaptation of large segmentation models for specific vision tasks, offering a parameter-efficient solution that reduces computational resource requirements.
The paper tackles the problem of adapting the Segment Anything Model 3 (SAM3) for downstream tasks at low cost by introducing SAM3-UNet, which outperforms prior methods like SAM2-UNet and other state-of-the-art approaches in tasks such as mirror detection and salient object detection, while using less than 6 GB of GPU memory during training with a batch size of 12.
In this paper, we introduce SAM3-UNet, a simplified variant of Segment Anything Model 3 (SAM3), designed to adapt SAM3 for downstream tasks at a low cost. Our SAM3-UNet consists of three components: a SAM3 image encoder, a simple adapter for parameter-efficient fine-tuning, and a lightweight U-Net-style decoder. Preliminary experiments on multiple tasks, such as mirror detection and salient object detection, demonstrate that the proposed SAM3-UNet outperforms the prior SAM2-UNet and other state-of-the-art methods, while requiring less than 6 GB of GPU memory during training with a batch size of 12. The code is publicly available at https://github.com/WZH0120/SAM3-UNet.