Reviving Iterative Training with Mask Guidance for Interactive Segmentation
This work addresses the problem of high computational cost for interactive segmentation, making it more deployable on mobile frameworks, though it is incremental as it builds on existing feedforward approaches.
The paper tackles the computational inefficiency of inference-time optimization in click-based interactive segmentation by proposing a simple feedforward model that uses previous masks, achieving state-of-the-art results without additional optimization. It finds that training on a combination of COCO and LVIS datasets yields superior performance.
Recent works on click-based interactive segmentation have demonstrated state-of-the-art results by using various inference-time optimization schemes. These methods are considerably more computationally expensive compared to feedforward approaches, as they require performing backward passes through a network during inference and are hard to deploy on mobile frameworks that usually support only forward passes. In this paper, we extensively evaluate various design choices for interactive segmentation and discover that new state-of-the-art results can be obtained without any additional optimization schemes. Thus, we propose a simple feedforward model for click-based interactive segmentation that employs the segmentation masks from previous steps. It allows not only to segment an entirely new object, but also to start with an external mask and correct it. When analyzing the performance of models trained on different datasets, we observe that the choice of a training dataset greatly impacts the quality of interactive segmentation. We find that the models trained on a combination of COCO and LVIS with diverse and high-quality annotations show performance superior to all existing models. The code and trained models are available at https://github.com/saic-vul/ritm_interactive_segmentation.