Learning to Forget with Information Divergence Reweighted Objectives for Noisy Labels
This addresses the problem of noisy labels in machine learning for practitioners, offering an incremental improvement over existing methods.
The paper tackles learning with noisy labels by introducing ANTIDOTE, an objective that adaptively reduces the influence of noisy samples, and shows it outperforms leading comparable losses in various noise settings with time complexity close to standard cross-entropy.
We introduce ANTIDOTE, a new class of objectives for learning under noisy labels which are defined in terms of a relaxation over an information-divergence neighborhood. Using convex duality, we provide a reformulation as an adversarial training method that has similar computational cost to training with standard cross-entropy loss. We show that our approach adaptively reduces the influence of the samples with noisy labels during learning, exhibiting a behavior that is analogous to forgetting those samples. ANTIDOTE is effective in practical environments where label noise is inherent in the training data or where an adversary can alter the training labels. Extensive empirical evaluations on different levels of symmetric, asymmetric, human annotation, and real-world label noise show that ANTIDOTE outperforms leading comparable losses in the field and enjoys a time complexity that is very close to that of the standard cross entropy loss.