KDAS: Knowledge Distillation via Attention Supervision Framework for Polyp Segmentation
This work addresses the need for compact, efficient models in medical imaging for practical industry applications, though it is incremental as it builds on existing knowledge distillation techniques.
The paper tackles the challenge of high computational cost in polyp segmentation models by proposing KDAS, a knowledge distillation framework with attention supervision and a Symmetrical Guiding Module, which achieves competitive results with state-of-the-art methods while using fewer parameters.
Polyp segmentation, a contentious issue in medical imaging, has seen numerous proposed methods aimed at improving the quality of segmented masks. While current state-of-the-art techniques yield impressive results, the size and computational cost of these models create challenges for practical industry applications. To address this challenge, we present KDAS, a Knowledge Distillation framework that incorporates attention supervision, and our proposed Symmetrical Guiding Module. This framework is designed to facilitate a compact student model with fewer parameters, allowing it to learn the strengths of the teacher model and mitigate the inconsistency between teacher features and student features, a common challenge in Knowledge Distillation, via the Symmetrical Guiding Module. Through extensive experiments, our compact models demonstrate their strength by achieving competitive results with state-of-the-art methods, offering a promising approach to creating compact models with high accuracy for polyp segmentation and in the medical imaging field. The implementation is available on https://github.com/huyquoctrinh/KDAS.