Adapting Foundation Models for Few-Shot Medical Image Segmentation: Actively and Sequentially
This work addresses the problem of reliable model adaptation for medical image segmentation with few annotated samples, offering a novel approach to reduce negative transfer in domain adaptation.
The paper tackled the challenge of adapting foundation models for few-shot medical image segmentation with large domain gaps, proposing an active and sequential framework that achieved average gains of 27.75% on MRI and 7.52% on CT datasets in Dice score compared to state-of-the-art methods.
Recent advances in foundation models have brought promising results in computer vision, including medical image segmentation. Fine-tuning foundation models on specific low-resource medical tasks has become a standard practice. However, ensuring reliable and robust model adaptation when the target task has a large domain gap and few annotated samples remains a challenge. Previous few-shot domain adaptation (FSDA) methods seek to bridge the distribution gap between source and target domains by utilizing auxiliary data. The selection and scheduling of auxiliaries are often based on heuristics, which can easily cause negative transfer. In this work, we propose an Active and Sequential domain AdaPtation (ASAP) framework for dynamic auxiliary dataset selection in FSDA. We formulate FSDA as a multi-armed bandit problem and derive an efficient reward function to prioritize training on auxiliary datasets that align closely with the target task, through a single-round fine-tuning. Empirical validation on diverse medical segmentation datasets demonstrates that our method achieves favorable segmentation performance, significantly outperforming the state-of-the-art FSDA methods, achieving an average gain of 27.75% on MRI and 7.52% on CT datasets in Dice score. Code is available at the git repository: https://github.com/techicoco/ASAP.