SELC: Self-Ensemble Label Correction Improves Learning with Noisy Labels
This addresses the issue of poor generalization in deep learning due to noisy labels, offering an incremental improvement over existing methods.
The paper tackles the problem of deep neural networks overfitting noisy labels by introducing Self-Ensemble Label Correction (SELC), which uses ensemble predictions from an exponential moving average to progressively correct labels, achieving more promising and stable results compared to state-of-the-art methods across various noise types.
Deep neural networks are prone to overfitting noisy labels, resulting in poor generalization performance. To overcome this problem, we present a simple and effective method self-ensemble label correction (SELC) to progressively correct noisy labels and refine the model. We look deeper into the memorization behavior in training with noisy labels and observe that the network outputs are reliable in the early stage. To retain this reliable knowledge, SELC uses ensemble predictions formed by an exponential moving average of network outputs to update the original noisy labels. We show that training with SELC refines the model by gradually reducing supervision from noisy labels and increasing supervision from ensemble predictions. Despite its simplicity, compared with many state-of-the-art methods, SELC obtains more promising and stable results in the presence of class-conditional, instance-dependent, and real-world label noise. The code is available at https://github.com/MacLLL/SELC.