Test-Time Adaptation by Causal Trimming
This addresses the issue of model reliability in real-world applications where data distributions shift, though it appears incremental as it builds on existing test-time adaptation methods.
The paper tackles the problem of model performance degradation under distribution shifts by introducing Test-time Adaptation by Causal Trimming (TACT), which identifies and removes non-causal features from representations to improve robustness, and it consistently outperforms state-of-the-art methods by a significant margin.
Test-time adaptation aims to improve model robustness under distribution shifts by adapting models with access to unlabeled target samples. A primary cause of performance degradation under such shifts is the model's reliance on features that lack a direct causal relationship with the prediction target. We introduce Test-time Adaptation by Causal Trimming (TACT), a method that identifies and removes non-causal components from representations for test distributions. TACT applies data augmentations that preserve causal features while varying non-causal ones. By analyzing the changes in the representations using Principal Component Analysis, TACT identifies the highest variance directions associated with non-causal features. It trims the representations by removing their projections on the identified directions, and uses the trimmed representations for the predictions. During adaptation, TACT continuously tracks and refines these directions to get a better estimate of non-causal features. We theoretically analyze the effectiveness of this approach and empirically validate TACT on real-world out-of-distribution benchmarks. TACT consistently outperforms state-of-the-art methods by a significant margin.