DST: Data Selection and joint Training for Learning with Noisy Labels
This addresses the challenge of noisy labels in automated annotation for deep learning, offering an incremental improvement over existing methods.
The paper tackles the problem of training deep neural networks with noisy labels by proposing a Data Selection and joint Training (DST) method that dynamically divides data into correctly labeled, correctly predicted, and wrong sets, achieving comparable or superior performance to state-of-the-art methods on datasets like CIFAR-10, CIFAR-100, and Clothing1M.
Training a deep neural network heavily relies on a large amount of training data with accurate annotations. To alleviate this problem, various methods have been proposed to annotate the data automatically. However, automatically generating annotations will inevitably yields noisy labels. In this paper, we propose a Data Selection and joint Training (DST) method to automatically select training samples with accurate annotations. Specifically, DST fits a mixture model according to the original annotation as well as the predicted label for each training sample, and the mixture model is utilized to dynamically divide the training dataset into a correctly labeled dataset, a correctly predicted set and a wrong dataset. Then, DST is trained with these datasets in a supervised manner. Due to confirmation bias problem, we train the two networks alternately, and each network is tasked to establish the data division to teach another network. For each iteration, the correctly labeled and predicted labels are reweighted respectively by the probabilities from the mixture model, and a uniform distribution is used to generate the probabilities of the wrong samples. Experiments on CIFAR-10, CIFAR-100 and Clothing1M demonstrate that DST is the comparable or superior to the state-of-the-art methods.