ZeroSiam: An Efficient Siamese for Test-Time Entropy Optimization without Collapse
This addresses a critical issue in test-time adaptation for machine learning models, preventing collapse solutions that can degrade performance, though it appears incremental as it builds on existing entropy minimization methods.
The paper tackles the problem of test-time entropy minimization leading to collapsed solutions, such as constant one-hot outputs, by introducing ZeroSiam, an efficient asymmetric Siamese architecture that prevents collapse through asymmetric divergence alignment, achieving stable performance with negligible overhead across vision adaptation and large language model reasoning tasks.
Test-time entropy minimization helps adapt a model to novel environments and incentivize its reasoning capability, unleashing the model's potential during inference by allowing it to evolve and improve in real-time using its own predictions, achieving promising performance. However, pure entropy minimization can favor non-generalizable shortcuts, such as inflating the logit norm and driving all predictions to a dominant class to reduce entropy, risking collapsed solutions (e.g., constant one-hot outputs) that trivially minimize the objective without meaningful learning. In this paper, we introduce ZeroSiam, an efficient asymmetric Siamese architecture tailored for test-time entropy minimization. ZeroSiam prevents collapse through asymmetric divergence alignment, which is efficiently achieved by a learnable predictor and a stop-gradient operator before the classifier. We provide empirical and theoretical evidence that ZeroSiam not only prevents collapse solutions, but also absorbs and regularizes biased learning signals, enhancing performance even when no collapse occurs. Despite its simplicity, extensive results show that ZeroSiam performs more stably over prior methods using negligible overhead, demonstrating efficacy on both vision adaptation and large language model reasoning tasks across challenging test scenarios and diverse models, including tiny models that are particularly collapse-prone.