DC-TTA: Divide-and-Conquer Framework for Test-Time Adaptation of Interactive Segmentation
This work addresses limitations in interactive segmentation for specialized domains, offering an incremental improvement by adapting SAM on a per-sample basis.
The paper tackles the problem of interactive segmentation in complex scenarios like camouflaged objects by proposing DC-TTA, a test-time adaptation framework that partitions user clicks into subsets for localized updates, resulting in significant performance improvements over SAM and conventional methods with fewer interactions and better accuracy.
Interactive segmentation (IS) allows users to iteratively refine object boundaries with minimal cues, such as positive and negative clicks. While the Segment Anything Model (SAM) has garnered attention in the IS community for its promptable segmentation capabilities, it often struggles in specialized domains or when handling complex scenarios (e.g., camouflaged or multi-part objects). To overcome these challenges, we propose DC-TTA, a novel test-time adaptation (TTA) framework that adapts SAM on a per-sample basis by leveraging user interactions as supervision. Instead of forcing a single model to incorporate all user clicks at once, DC-TTA partitions the clicks into more coherent subsets, each processed independently via TTA with a separated model. This Divide-and-Conquer strategy reduces conflicts among diverse cues and enables more localized updates. Finally, we merge the adapted models to form a unified predictor that integrates the specialized knowledge from each subset. Experimental results across various benchmarks demonstrate that DC-TTA significantly outperforms SAM's zero-shot results and conventional TTA methods, effectively handling complex tasks such as camouflaged object segmentation with fewer interactions and improved accuracy.