From Adaptation to Generalization: Adaptive Visual Prompting for Medical Image Segmentation
For medical image segmentation, APEX addresses the critical problem of intra- and inter-domain variability by providing a plug-and-play prompting solution that enhances generalization without model updates.
APEX introduces an adaptive visual prompting framework that retrieves input-specific prompts from a learnable memory, improving generalization across seen and unseen domains in medical image segmentation. It achieves significant performance gains over existing methods on two medical segmentation tasks.
Visual prompting has emerged as a powerful method for adapting pre-trained models to new domains without updating model parameters. However, existing prompting methods typically optimize a single prompt per domain and apply it uniformly to all inputs, limiting their ability to generalize under intra and inter-domain variability, which is especially critical in the medical field. To address this, we propose APEX, an Adaptive Prompt EXtraction framework that retrieves input-specific prompts from a learnable prompt memory. The memory stores diverse, domain-discriminative prompt representations and is queried via domain features extracted from the Fourier spectrum. To learn robust and discriminative domain features, we introduce a novel Low-Frequency Feature Contrastive (LFC) learning framework that clusters representations from the same domain while separating those from different domains. Extensive experiments on two medical segmentation tasks demonstrate that APEX significantly improves generalization across both seen and unseen domains. Furthermore, it complements any existing backbones and consistently enhances performance, confirming its effectiveness as a plug-and-play prompting solution in medical fields. The code is available at https://github.com/cetinkayaevren/apex/