Private and Stable Test-Time Adaptation with Differential Privacy
For practitioners deploying TTA in sensitive domains, this work provides the first privacy-preserving framework with empirical validation, though the methods are incremental adaptations of existing TTA techniques.
This paper applies differential privacy to test-time adaptation methods, showing that per-sample gradient clipping and Gaussian noise can provide privacy with minimal accuracy loss on ImageNet-C, and even improve adaptation stability in low-privacy regimes.
Test-time adaptation (TTA) can reduce error on new and different data by updating the model on these inputs during inference. However, these updates raise the issue of privacy w.r.t. the testing data, because the model parameters now depend on all past inputs. To control this privacy risk, we cast multiple popular TTA methods (Tent, EATA, SAR, DeYO, and COME) into differential privacy (DP) forms that apply per-sample gradient clipping and Gaussian noise for all updates. On ImageNet-C, our DP-TTA methods provide adequate privacy at small cost to accuracy, and in the low-privacy regime the clipping mechanism of DP can even improve the accuracy and stability of adaptation in the continual setting. These improvements to privacy and accuracy come at only modest computational overhead. These first results on private TTA raise awareness of the issue, inform the development of more private test-time updates, and identify per-sample clipping as an effective technique for improving the accuracy and stability of adaptation.