Mitigating the Bias in the Model for Continual Test-Time Adaptation
This work addresses a specific issue in continual adaptation for machine learning models, offering an incremental improvement to existing methods in this domain.
The paper tackles the problem of biased predictions in Continual Test-Time Adaptation, where models become over-confident and inaccurate as they adapt to changing target domains, and proposes a method that improves performance by aligning target features with source prototypes and using class-wise target prototypes, achieving noteworthy performance gains without significant adaptation time overhead.
Continual Test-Time Adaptation (CTA) is a challenging task that aims to adapt a source pre-trained model to continually changing target domains. In the CTA setting, a model does not know when the target domain changes, thus facing a drastic change in the distribution of streaming inputs during the test-time. The key challenge is to keep adapting the model to the continually changing target domains in an online manner. We find that a model shows highly biased predictions as it constantly adapts to the chaining distribution of the target data. It predicts certain classes more often than other classes, making inaccurate over-confident predictions. This paper mitigates this issue to improve performance in the CTA scenario. To alleviate the bias issue, we make class-wise exponential moving average target prototypes with reliable target samples and exploit them to cluster the target features class-wisely. Moreover, we aim to align the target distributions to the source distribution by anchoring the target feature to its corresponding source prototype. With extensive experiments, our proposed method achieves noteworthy performance gain when applied on top of existing CTA methods without substantial adaptation time overhead.