Adaptive and Balanced Re-initialization for Long-timescale Continual Test-time Domain Adaptation
This work addresses the challenge of adapting models to non-stationary environments over time, which is incremental as it builds on existing CTTA methods.
The paper tackles the problem of maintaining model performance in continual test-time domain adaptation over long timescales by proposing an adaptive and balanced re-initialization method, achieving superior results on benchmarks.
Continual test-time domain adaptation (CTTA) aims to adjust models so that they can perform well over time across non-stationary environments. While previous methods have made considerable efforts to optimize the adaptation process, a crucial question remains: Can the model adapt to continually changing environments over a long time? In this work, we explore facilitating better CTTA in the long run using a re-initialization (or reset) based method. First, we observe that the long-term performance is associated with the trajectory pattern in label flip. Based on this observed correlation, we propose a simple yet effective policy, Adaptive-and-Balanced Re-initialization (ABR), towards preserving the model's long-term performance. In particular, ABR performs weight re-initialization using adaptive intervals. The adaptive interval is determined based on the change in label flip. The proposed method is validated on extensive CTTA benchmarks, achieving superior performance.