Deep k-NN for Noisy Labels
This addresses the issue of noisy labels that degrade model performance for machine learning practitioners, though it appears incremental as it builds on existing filtering techniques.
The paper tackles the problem of noisy labels in machine learning training data by showing that a simple k-nearest neighbor-based filtering approach on logit layers can remove mislabeled examples, resulting in more accurate models than recent methods, with empirical evidence supporting its efficacy.
Modern machine learning models are often trained on examples with noisy labels that hurt performance and are hard to identify. In this paper, we provide an empirical study showing that a simple $k$-nearest neighbor-based filtering approach on the logit layer of a preliminary model can remove mislabeled training data and produce more accurate models than many recently proposed methods. We also provide new statistical guarantees into its efficacy.