Beyond Visual Cues: Leveraging General Semantics as Support for Few-Shot Segmentation
This work addresses the challenge of segmenting novel classes with limited samples in computer vision, offering a novel solution that could enhance applications in domains like medical imaging or autonomous driving, though it appears incremental by building on existing meta-learning paradigms.
The paper tackles the problem of few-shot segmentation by addressing intra-class variations in visual representations, proposing a language-driven approach that leverages general semantics from attribute descriptions to provide unbiased meta guidance, resulting in new state-of-the-art performance with clear margin improvements over existing methods.
Few-shot segmentation (FSS) aims to segment novel classes under the guidance of limited support samples by a meta-learning paradigm. Existing methods mainly mine references from support images as meta guidance. However, due to intra-class variations among visual representations, the meta information extracted from support images cannot produce accurate guidance to segment untrained classes. In this paper, we argue that the references from support images may not be essential, the key to the support role is to provide unbiased meta guidance for both trained and untrained classes. We then introduce a Language-Driven Attribute Generalization (LDAG) architecture to utilize inherent target property language descriptions to build robust support strategy. Specifically, to obtain an unbiased support representation, we design a Multi-attribute Enhancement (MaE) module, which produces multiple detailed attribute descriptions of the target class through Large Language Models (LLMs), and then builds refined visual-text prior guidance utilizing multi-modal matching. Meanwhile, due to text-vision modal shift, attribute text struggles to promote visual feature representation, we design a Multi-modal Attribute Alignment (MaA) to achieve cross-modal interaction between attribute texts and visual feature. Experiments show that our proposed method outperforms existing approaches by a clear margin and achieves the new state-of-the art performance. The code will be released.