Robust Temporal Ensembling for Learning with Noisy Labels
This addresses the challenge of noisy labels in real-world datasets for machine learning practitioners, offering a robust method without relying on label filtering or fixing.
The paper tackles the problem of training deep neural networks with noisy labels, which can degrade supervised learning, by proposing robust temporal ensembling (RTE) that combines robust loss with semi-supervised regularization. It achieves state-of-the-art performance on datasets like CIFAR-10, CIFAR-100, ImageNet, WebVision, and Food-101N, and shows competitive corruption robustness with a mean corruption error of 13.50% at 80% noise ratio versus 26.9% with standard methods on clean data.
Successful training of deep neural networks with noisy labels is an essential capability as most real-world datasets contain some amount of mislabeled data. Left unmitigated, label noise can sharply degrade typical supervised learning approaches. In this paper, we present robust temporal ensembling (RTE), which combines robust loss with semi-supervised regularization methods to achieve noise-robust learning. We demonstrate that RTE achieves state-of-the-art performance across the CIFAR-10, CIFAR-100, ImageNet, WebVision, and Food-101N datasets, while forgoing the recent trend of label filtering and/or fixing. Finally, we show that RTE also retains competitive corruption robustness to unforeseen input noise using CIFAR-10-C, obtaining a mean corruption error (mCE) of 13.50% even in the presence of an 80% noise ratio, versus 26.9% mCE with standard methods on clean data.