ETAGE: Enhanced Test Time Adaptation with Integrated Entropy and Gradient Norms for Robust Model Performance
This work addresses robust model performance in test time adaptation for deep learning, but it is incremental as it refines existing approaches like PLPD.
The paper tackled the problem of test time adaptation (TTA) for deep learning models handling unseen test data, particularly in biased scenarios, by introducing ETAGE, which integrates entropy minimization with gradient norms and PLPD to enhance sample selection and adaptation, resulting in outperforming existing TTA techniques on CIFAR-10-C and CIFAR-100-C datasets.
Test time adaptation (TTA) equips deep learning models to handle unseen test data that deviates from the training distribution, even when source data is inaccessible. While traditional TTA methods often rely on entropy as a confidence metric, its effectiveness can be limited, particularly in biased scenarios. Extending existing approaches like the Pseudo Label Probability Difference (PLPD), we introduce ETAGE, a refined TTA method that integrates entropy minimization with gradient norms and PLPD, to enhance sample selection and adaptation. Our method prioritizes samples that are less likely to cause instability by combining high entropy with high gradient norms out of adaptation, thus avoiding the overfitting to noise often observed in previous methods. Extensive experiments on CIFAR-10-C and CIFAR-100-C datasets demonstrate that our approach outperforms existing TTA techniques, particularly in challenging and biased scenarios, leading to more robust and consistent model performance across diverse test scenarios. The codebase for ETAGE is available on https://github.com/afsharshamsi/ETAGE.