Learning Deep Networks from Noisy Labels with Dropout Regularization
This addresses the issue of poor classifier performance due to label noise for researchers and practitioners using large, noisy datasets, representing an incremental improvement over existing methods.
The paper tackles the problem of training deep neural networks on datasets with unreliable labels, such as from crowdsourcing or social media, by proposing a technique that augments a standard network with a softmax layer to model label noise and uses dropout regularization to learn a non-trivial noise model, resulting in outperforming state-of-the-art methods on noisy versions of CIFAR-10 and MNIST datasets.
Large datasets often have unreliable labels-such as those obtained from Amazon's Mechanical Turk or social media platforms-and classifiers trained on mislabeled datasets often exhibit poor performance. We present a simple, effective technique for accounting for label noise when training deep neural networks. We augment a standard deep network with a softmax layer that models the label noise statistics. Then, we train the deep network and noise model jointly via end-to-end stochastic gradient descent on the (perhaps mislabeled) dataset. The augmented model is overdetermined, so in order to encourage the learning of a non-trivial noise model, we apply dropout regularization to the weights of the noise model during training. Numerical experiments on noisy versions of the CIFAR-10 and MNIST datasets show that the proposed dropout technique outperforms state-of-the-art methods.