SPT: Sequence Prompt Transformer for Interactive Image Segmentation
This addresses a practical need in real-world applications where sequential images of the same object must be segmented, representing a novel approach beyond single-image methods.
The paper tackles the problem of segmenting a series of images featuring the same target object in interactive segmentation, proposing Sequence Prompt Transformer (SPT) which processes sequential image information and achieves state-of-the-art results across multiple benchmark datasets.
Interactive segmentation aims to extract objects of interest from an image based on user-provided clicks. In real-world applications, there is often a need to segment a series of images featuring the same target object. However, existing methods typically process one image at a time, failing to consider the sequential nature of the images. To overcome this limitation, we propose a novel method called Sequence Prompt Transformer (SPT), the first to utilize sequential image information for interactive segmentation. Our model comprises two key components: (1) Sequence Prompt Transformer (SPT) for acquiring information from sequence of images, clicks and masks to improve accurate. (2) Top-k Prompt Selection (TPS) selects precise prompts for SPT to further enhance the segmentation effect. Additionally, we create the ADE20K-Seq benchmark to better evaluate model performance. We evaluate our approach on multiple benchmark datasets and show that our model surpasses state-of-the-art methods across all datasets.