Temporal Test-Time Adaptation with State-Space Models
This addresses the challenge of adapting models to evolving real-world distribution shifts, which is crucial for maintaining performance in deployed AI systems, representing an incremental improvement over existing test-time adaptation methods.
The paper tackles the problem of performance decay in deployed models due to gradual temporal distribution shifts, proposing STAD, a Bayesian filtering method that adapts models by learning time-varying dynamics in hidden features and inferring class prototypes without labels, achieving strong results in handling small batch sizes and label shift in real-world experiments.
Distribution shifts between training and test data are inevitable over the lifecycle of a deployed model, leading to performance decay. Adapting a model on test samples can help mitigate this drop in performance. However, most test-time adaptation methods have focused on synthetic corruption shifts, leaving a variety of distribution shifts underexplored. In this paper, we focus on distribution shifts that evolve gradually over time, which are common in the wild but challenging for existing methods, as we show. To address this, we propose STAD, a Bayesian filtering method that adapts a deployed model to temporal distribution shifts by learning the time-varying dynamics in the last set of hidden features. Without requiring labels, our model infers time-evolving class prototypes that act as a dynamic classification head. Through experiments on real-world temporal distribution shifts, we show that our method excels in handling small batch sizes and label shift.