Annotation-Efficient Active Test-Time Adaptation with Conformal Prediction
This work addresses the challenge of efficiently using human annotation budgets for model adaptation under domain shift, representing an incremental advance in ATTA methods.
The paper tackles the problem of low data selection efficiency in Active Test-Time Adaptation (ATTA) by proposing CPATTA, which uses conformal prediction for principled uncertainty estimation, resulting in a consistent accuracy improvement of around 5% over state-of-the-art methods.
Active Test-Time Adaptation (ATTA) improves model robustness under domain shift by selectively querying human annotations at deployment, but existing methods use heuristic uncertainty measures and suffer from low data selection efficiency, wasting human annotation budget. We propose Conformal Prediction Active TTA (CPATTA), which first brings principled, coverage-guaranteed uncertainty into ATTA. CPATTA employs smoothed conformal scores with a top-K certainty measure, an online weight-update algorithm driven by pseudo coverage, a domain-shift detector that adapts human supervision, and a staged update scheme balances human-labeled and model-labeled data. Extensive experiments demonstrate that CPATTA consistently outperforms the state-of-the-art ATTA methods by around 5% in accuracy. Our code and datasets are available at https://github.com/tingyushi/CPATTA.