Adaptive Cascading Network for Continual Test-Time Adaptation
This addresses the challenge of adapting pre-trained models to unlabelled target domains at test time for applications in dynamic, real-world scenarios, representing an incremental improvement over existing methods.
The paper tackles the problem of continual test-time adaptation by proposing a cascading paradigm that updates both the feature extractor and classifier simultaneously, mitigating mismatch and enabling long-term adaptation, with experiments showing superiority in tasks like image classification, text classification, and speech recognition.
We study the problem of continual test-time adaption where the goal is to adapt a source pre-trained model to a sequence of unlabelled target domains at test time. Existing methods on test-time training suffer from several limitations: (1) Mismatch between the feature extractor and classifier; (2) Interference between the main and self-supervised tasks; (3) Lack of the ability to quickly adapt to the current distribution. In light of these challenges, we propose a cascading paradigm that simultaneously updates the feature extractor and classifier at test time, mitigating the mismatch between them and enabling long-term model adaptation. The pre-training of our model is structured within a meta-learning framework, thereby minimizing the interference between the main and self-supervised tasks and encouraging fast adaptation in the presence of limited unlabelled data. Additionally, we introduce innovative evaluation metrics, average accuracy and forward transfer, to effectively measure the model's adaptation capabilities in dynamic, real-world scenarios. Extensive experiments and ablation studies demonstrate the superiority of our approach in a range of tasks including image classification, text classification, and speech recognition.