When Test-Time Adaptation Meets Self-Supervised Models
This addresses the challenge of adapting self-supervised models to dynamic environments for machine learning practitioners, but it is incremental as it builds on existing TTA and SSL methods.
The paper tackles the problem of test-time adaptation (TTA) for self-supervised learning (SSL) models, which struggle with low source accuracy, by proposing a collaborative framework that integrates SSL and TTA using contrastive learning and knowledge distillation, achieving competitive performance without relying on source pretraining.
Training on test-time data enables deep learning models to adapt to dynamic environmental changes, enhancing their practical applicability. Online adaptation from source to target domains is promising but it remains highly reliant on the performance of source pretrained model. In this paper, we investigate whether test-time adaptation (TTA) methods can continuously improve models trained via self-supervised learning (SSL) without relying on source pretraining. We introduce a self-supervised TTA protocol after observing that existing TTA approaches struggle when directly applied to self-supervised models with low accuracy on the source domain. Furthermore, we propose a collaborative learning framework that integrates SSL and TTA models, leveraging contrastive learning and knowledge distillation for stepwise representation refinement. We validate our method on diverse self-supervised models, including DINO, MoCo, and iBOT, across TTA benchmarks. Extensive experiments validate the effectiveness of our approach in SSL, showing that it achieves competitive performance even without source pretraining.