ASAP: Unsupervised Post-training with Label Distribution Shift Adaptive Learning Rate
This addresses the challenge of adapting models to changing label distributions in real-world applications, though it is incremental as it builds on existing adaptation techniques.
The paper tackles the problem of online label shift in machine learning models by proposing ASAP, an unsupervised post-training method that dynamically adjusts the learning rate based on cosine distance between outputs, resulting in improved accuracy and efficiency across various datasets and shift scenarios.
In real-world applications, machine learning models face online label shift, where label distributions change over time. Effective adaptation requires careful learning rate selection: too low slows adaptation and too high causes instability. We propose ASAP (Adaptive Shift Aware Post-training), which dynamically adjusts the learning rate by computing the cosine distance between current and previous unlabeled outputs and mapping it within a bounded range. ASAP requires no labels, model ensembles, or past inputs, using only the previous softmax output for fast, lightweight adaptation. Experiments across multiple datasets and shift scenarios show ASAP consistently improves accuracy and efficiency, making it practical for unsupervised model adaptation.