Back to the Source: Diffusion-Driven Test-Time Adaptation
This addresses the challenge of model degradation in test-time adaptation for computer vision, offering a more robust alternative to existing model adaptation methods, though it is incremental in its approach.
The paper tackles the problem of test-time adaptation for models facing domain shifts by proposing a method that adapts target data to the source domain using a generative diffusion model, achieving robustness across various corruptions and data regimes on ImageNet-C.
Test-time adaptation harnesses test inputs to improve the accuracy of a model trained on source data when tested on shifted target data. Existing methods update the source model by (re-)training on each target domain. While effective, re-training is sensitive to the amount and order of the data and the hyperparameters for optimization. We instead update the target data, by projecting all test inputs toward the source domain with a generative diffusion model. Our diffusion-driven adaptation method, DDA, shares its models for classification and generation across all domains. Both models are trained on the source domain, then fixed during testing. We augment diffusion with image guidance and self-ensembling to automatically decide how much to adapt. Input adaptation by DDA is more robust than prior model adaptation approaches across a variety of corruptions, architectures, and data regimes on the ImageNet-C benchmark. With its input-wise updates, DDA succeeds where model adaptation degrades on too little data in small batches, dependent data in non-uniform order, or mixed data with multiple corruptions.