Progressive Test Time Energy Adaptation for Medical Image Segmentation
This addresses the problem of distribution shifts in medical imaging for clinicians, offering a practical solution without requiring multiple passes through target data.
The paper tackles the challenge of maintaining segmentation model performance across diverse medical datasets by proposing a progressive test-time energy adaptation approach, which consistently outperforms baselines on eight public MRI and X-ray datasets.
We propose a model-agnostic, progressive test-time energy adaptation approach for medical image segmentation. Maintaining model performance across diverse medical datasets is challenging, as distribution shifts arise from inconsistent imaging protocols and patient variations. Unlike domain adaptation methods that require multiple passes through target data - impractical in clinical settings - our approach adapts pretrained models progressively as they process test data. Our method leverages a shape energy model trained on source data, which assigns an energy score at the patch level to segmentation maps: low energy represents in-distribution (accurate) shapes, while high energy signals out-of-distribution (erroneous) predictions. By minimizing this energy score at test time, we refine the segmentation model to align with the target distribution. To validate the effectiveness and adaptability, we evaluated our framework on eight public MRI (bSSFP, T1- and T2-weighted) and X-ray datasets spanning cardiac, spinal cord, and lung segmentation. We consistently outperform baselines both quantitatively and qualitatively.