Channel Attention-Guided Cross-Modal Knowledge Distillation for Referring Image Segmentation
For practitioners deploying RIS models on resource-constrained devices, this method enables smaller models to approach the performance of larger ones.
The paper proposes a channel attention-guided cross-modal knowledge distillation method for referring image segmentation that transfers fine-grained cross-modal correlations from a teacher to a student network, achieving significant performance improvement without adding inference parameters.
Referring image segmentation (RIS) requires accurate segmentation of target regions in images according to language descriptions, which is a cross-modal task integrating vision and language. Existing RIS methods typically employ large-scale vision and language encoding models to improve performance, but their enormous parameter size severely restricts deployment in scenarios with limited computing resources. To solve this problem, this paper proposes a channel attention-guided cross-modal knowledge distillation method, which transfers the high-order fine-grained correlations between vision and language learned by the teacher network, as well as the correlations between semantic components represented by each channel, to the student network. Compared with the traditional pixel-wise relational distillation, this method not only enables the student to learn the knowledge of the teacher, but also retains part of its independent learning ability, alleviating the transfer of learning bias. Experimental results on two public datasets show that the proposed distillation method does not introduce additional parameters during inference and can achieve significant performance improvement for the student model.