Pick the Best Pre-trained Model: Towards Transferability Estimation for Medical Image Segmentation
This work addresses the challenge of efficient model reuse for medical image segmentation, which is incremental as it improves upon existing transferability estimation methods in a specific domain.
The paper tackles the problem of selecting the best pre-trained model for medical image segmentation by proposing a new transferability estimation method that considers class consistency and feature variety, achieving state-of-the-art performance in experiments.
Transfer learning is a critical technique in training deep neural networks for the challenging medical image segmentation task that requires enormous resources. With the abundance of medical image data, many research institutions release models trained on various datasets that can form a huge pool of candidate source models to choose from. Hence, it's vital to estimate the source models' transferability (i.e., the ability to generalize across different downstream tasks) for proper and efficient model reuse. To make up for its deficiency when applying transfer learning to medical image segmentation, in this paper, we therefore propose a new Transferability Estimation (TE) method. We first analyze the drawbacks of using the existing TE algorithms for medical image segmentation and then design a source-free TE framework that considers both class consistency and feature variety for better estimation. Extensive experiments show that our method surpasses all current algorithms for transferability estimation in medical image segmentation. Code is available at https://github.com/EndoluminalSurgicalVision-IMR/CCFV