Adapting Segment Anything Model for Unseen Object Instance Segmentation
This work addresses a crucial challenge for autonomous robots in unstructured environments, offering a data-efficient solution for segmenting unseen objects, though it is incremental as it builds upon the Segment Anything Model.
The paper tackles the problem of Unseen Object Instance Segmentation (UOIS) for autonomous robots by proposing UOIS-SAM, which integrates a Heatmap-based Prompt Generator and a Hierarchical Discrimination Network to adapt the Segment Anything Model, achieving state-of-the-art performance with only 10% of the training samples compared to previous methods.
Unseen Object Instance Segmentation (UOIS) is crucial for autonomous robots operating in unstructured environments. Previous approaches require full supervision on large-scale tabletop datasets for effective pretraining. In this paper, we propose UOIS-SAM, a data-efficient solution for the UOIS task that leverages SAM's high accuracy and strong generalization capabilities. UOIS-SAM integrates two key components: (i) a Heatmap-based Prompt Generator (HPG) to generate class-agnostic point prompts with precise foreground prediction, and (ii) a Hierarchical Discrimination Network (HDNet) that adapts SAM's mask decoder, mitigating issues introduced by the SAM baseline, such as background confusion and over-segmentation, especially in scenarios involving occlusion and texture-rich objects. Extensive experimental results on OCID, OSD, and additional photometrically challenging datasets including PhoCAL and HouseCat6D, demonstrate that, even using only 10% of the training samples compared to previous methods, UOIS-SAM achieves state-of-the-art performance in unseen object segmentation, highlighting its effectiveness and robustness in various tabletop scenes.