Enhancing Generalized Few-Shot Semantic Segmentation via Effective Knowledge Transfer
This work addresses the problem of segmenting both base and novel classes with limited data for computer vision researchers, representing an incremental advance in few-shot learning methods.
The paper tackles the distribution gap between base and novel classes in generalized few-shot semantic segmentation by proposing a prototype modulation module, classifier calibration module, and context consistency learning scheme, achieving significant state-of-the-art improvements on PASCAL-5^i and COCO-20^i datasets.
Generalized few-shot semantic segmentation (GFSS) aims to segment objects of both base and novel classes, using sufficient samples of base classes and few samples of novel classes. Representative GFSS approaches typically employ a two-phase training scheme, involving base class pre-training followed by novel class fine-tuning, to learn the classifiers for base and novel classes respectively. Nevertheless, distribution gap exists between base and novel classes in this process. To narrow this gap, we exploit effective knowledge transfer from base to novel classes. First, a novel prototype modulation module is designed to modulate novel class prototypes by exploiting the correlations between base and novel classes. Second, a novel classifier calibration module is proposed to calibrate the weight distribution of the novel classifier according to that of the base classifier. Furthermore, existing GFSS approaches suffer from a lack of contextual information for novel classes due to their limited samples, we thereby introduce a context consistency learning scheme to transfer the contextual knowledge from base to novel classes. Extensive experiments on PASCAL-5$^i$ and COCO-20$^i$ demonstrate that our approach significantly enhances the state of the art in the GFSS setting. The code is available at: https://github.com/HHHHedy/GFSS-EKT.