Gradient Alignment with Prototype Feature for Fully Test-time Adaptation
This addresses the challenge of maintaining model performance during adaptation in test-time adaptation, an incremental improvement for machine learning applications requiring adaptation to new data without labels.
The paper tackles the problem of inappropriate guidance from entropy minimization loss due to misclassified pseudo labels in test-time adaptation by proposing a Gradient Alignment with Prototype feature (GAP) regularizer, which significantly improves TTA methods across various datasets.
In context of Test-time Adaptation(TTA), we propose a regularizer, dubbed Gradient Alignment with Prototype feature (GAP), which alleviates the inappropriate guidance from entropy minimization loss from misclassified pseudo label. We developed a gradient alignment loss to precisely manage the adaptation process, ensuring that changes made for some data don't negatively impact the model's performance on other data. We introduce a prototype feature of a class as a proxy measure of the negative impact. To make GAP regularizer feasible under the TTA constraints, where model can only access test data without labels, we tailored its formula in two ways: approximating prototype features with weight vectors of the classifier, calculating gradient without back-propagation. We demonstrate GAP significantly improves TTA methods across various datasets, which proves its versatility and effectiveness.