Matching High-Dimensional Geometric Quantiles for Test-Time Adaptation of Transformers and Convolutional Networks Alike
This addresses the problem of distribution shift in test data for machine learning practitioners, offering a generic solution applicable to both convolutional and transformer networks, though it appears incremental by building on existing TTA methods.
The paper tackles test-time adaptation (TTA) for classifiers when test data distribution differs from training data, proposing an architecture-agnostic adapter network with a quantile loss that matches high-dimensional geometric quantiles, achieving improved performance on benchmarks like CIFAR10-C, CIFAR100-C, and TinyImageNet-C.
Test-time adaptation (TTA) refers to adapting a classifier for the test data when the probability distribution of the test data slightly differs from that of the training data of the model. To the best of our knowledge, most of the existing TTA approaches modify the weights of the classifier relying heavily on the architecture. It is unclear as to how these approaches are extendable to generic architectures. In this article, we propose an architecture-agnostic approach to TTA by adding an adapter network pre-processing the input images suitable to the classifier. This adapter is trained using the proposed quantile loss. Unlike existing approaches, we correct for the distribution shift by matching high-dimensional geometric quantiles. We prove theoretically that under suitable conditions minimizing quantile loss can learn the optimal adapter. We validate our approach on CIFAR10-C, CIFAR100-C and TinyImageNet-C by training both classic convolutional and transformer networks on CIFAR10, CIFAR100 and TinyImageNet datasets.