Learning to Detect Noisy Labels Using Model-Based Features
This work addresses label noise issues in scenarios like self-training and data annotation, offering a flexible, data-driven approach that is incremental over existing methods.
The paper tackles the problem of noisy labels in machine learning by proposing SENT, a method that trains a model to distinguish noisy from clean labels using model-based features, achieving improved performance on text classification and speech recognition tasks.
Label noise is ubiquitous in various machine learning scenarios such as self-labeling with model predictions and erroneous data annotation. Many existing approaches are based on heuristics such as sample losses, which might not be flexible enough to achieve optimal solutions. Meta learning based methods address this issue by learning a data selection function, but can be hard to optimize. In light of these pros and cons, we propose Selection-Enhanced Noisy label Training (SENT) that does not rely on meta learning while having the flexibility of being data-driven. SENT transfers the noise distribution to a clean set and trains a model to distinguish noisy labels from clean ones using model-based features. Empirically, on a wide range of tasks including text classification and speech recognition, SENT improves performance over strong baselines under the settings of self-training and label corruption.