MARS: a Multimodal Alignment and Ranking System for Few-Shot Segmentation
It addresses the lack of robust selection methods in few-shot segmentation, enabling rapid adaptation with minimal supervision for tasks like object segmentation.
The paper tackles the problem of few-shot segmentation by introducing MARS, a plug-and-play ranking system that uses multimodal cues to filter and merge mask proposals, achieving new state-of-the-art results on benchmarks like COCO-20i and Pascal-5i.
Few Shot Segmentation aims to segment novel object classes given only a handful of labeled examples, enabling rapid adaptation with minimal supervision. Current literature crucially lacks a selection method that goes beyond visual similarity between the query and example images, leading to suboptimal predictions. We present MARS, a plug-and-play ranking system that leverages multimodal cues to filter and merge mask proposals robustly. Starting from a set of mask predictions for a single query image, we score, filter, and merge them to improve results. Proposals are evaluated using multimodal scores computed at local and global levels. Extensive experiments on COCO-20i, Pascal-5i, LVIS-92i, and FSS-1000 demonstrate that integrating all four scoring components is crucial for robust ranking, validating our contribution. As MARS can be effortlessly integrated with various mask proposal systems, we deploy it across a wide range of top-performer methods and achieve new state-of-the-art results on multiple existing benchmarks. Code will be available upon acceptance.