Collaborative Label Correction via Entropy Thresholding
This addresses the issue of noisy labels in machine learning, which can degrade model performance, but it is incremental as it builds on existing entropy-based methods.
The paper tackles the problem of noisy labels in deep neural networks by proposing a Collaborative Label Correction (CLC) framework that uses entropy thresholding to identify reliable predictions for supervision, outperforming state-of-the-art methods in experiments on multiple benchmarks.
Deep neural networks (DNNs) have the capacity to fit extremely noisy labels nonetheless they tend to learn data with clean labels first and then memorize those with noisy labels. We examine this behavior in light of the Shannon entropy of the predictions and demonstrate the low entropy predictions determined by a given threshold are much more reliable as the supervision than the original noisy labels. It also shows the advantage in maintaining more training samples than previous methods. Then, we power this entropy criterion with the Collaborative Label Correction (CLC) framework to further avoid undesired local minimums of the single network. A range of experiments have been conducted on multiple benchmarks with both synthetic and real-world settings. Extensive results indicate that our CLC outperforms several state-of-the-art methods.