f-BRS: Rethinking Backpropagating Refinement for Interactive Segmentation
This work addresses the computational inefficiency in interactive segmentation for users needing fast and accurate object segmentation, representing an incremental improvement over existing backpropagating refinement schemes.
The paper tackles the problem of interactive segmentation where existing methods struggle with unknown objects despite extensive user input, and introduces f-BRS which achieves state-of-the-art performance on multiple datasets while reducing computational time per click by an order of magnitude compared to prior methods.
Deep neural networks have become a mainstream approach to interactive segmentation. As we show in our experiments, while for some images a trained network provides accurate segmentation result with just a few clicks, for some unknown objects it cannot achieve satisfactory result even with a large amount of user input. Recently proposed backpropagating refinement (BRS) scheme introduces an optimization problem for interactive segmentation that results in significantly better performance for the hard cases. At the same time, BRS requires running forward and backward pass through a deep network several times that leads to significantly increased computational budget per click compared to other methods. We propose f-BRS (feature backpropagating refinement scheme) that solves an optimization problem with respect to auxiliary variables instead of the network inputs, and requires running forward and backward pass just for a small part of a network. Experiments on GrabCut, Berkeley, DAVIS and SBD datasets set new state-of-the-art at an order of magnitude lower time per click compared to original BRS. The code and trained models are available at https://github.com/saic-vul/fbrs_interactive_segmentation .