Order-Aware Test-Time Adaptation: Leveraging Temporal Dynamics for Robust Streaming Inference
This addresses robust streaming inference for machine learning models by incorporating temporal order, offering an incremental improvement over existing TTA methods.
The paper tackled the problem of test-time adaptation (TTA) overlooking temporal dynamics in streaming data by introducing Order-Aware Test-Time Adaptation (OATTA), which improved accuracy by up to 6.35% across various domains.
Test-Time Adaptation (TTA) enables pre-trained models to adjust to distribution shift by learning from unlabeled test-time streams. However, existing methods typically treat these streams as independent samples, overlooking the supervisory signal inherent in temporal dynamics. To address this, we introduce Order-Aware Test-Time Adaptation (OATTA). We formulate test-time adaptation as a gradient-free recursive Bayesian estimation task, using a learned dynamic transition matrix as a temporal prior to refine the base model's predictions. To ensure safety in weakly structured streams, we introduce a likelihood-ratio gate (LLR) that reverts to the base predictor when temporal evidence is absent. OATTA is a lightweight, model-agnostic module that incurs negligible computational overhead. Extensive experiments across image classification, wearable and physiological signal analysis, and language sentiment analysis demonstrate its universality; OATTA consistently boosts established baselines, improving accuracy by up to 6.35%. Our findings establish that modeling temporal dynamics provides a critical, orthogonal signal beyond standard order-agnostic TTA approaches.