Annot-Mix: Learning with Noisy Class Labels from Multiple Annotators via a Mixup Extension
This addresses the challenge of noisy labels in machine learning, particularly for applications relying on crowd-sourced data, though it is an incremental improvement on existing mixup techniques.
The paper tackles the problem of training neural networks with noisy class labels from multiple annotators by extending mixup to handle multiple labels per instance, achieving superior performance over eight state-of-the-art methods on eleven datasets.
Training with noisy class labels impairs neural networks' generalization performance. In this context, mixup is a popular regularization technique to improve training robustness by making memorizing false class labels more difficult. However, mixup neglects that, typically, multiple annotators, e.g., crowdworkers, provide class labels. Therefore, we propose an extension of mixup, which handles multiple class labels per instance while considering which class label originates from which annotator. Integrated into our multi-annotator classification framework annot-mix, it performs superiorly to eight state-of-the-art approaches on eleven datasets with noisy class labels provided either by human or simulated annotators. Our code is publicly available through our repository at https://github.com/ies-research/annot-mix.