Towards a New Understanding of the Training of Neural Networks with Mislabeled Training Data
This addresses the issue of mislabeled training data for machine learning practitioners, but it appears incremental as it builds on existing Maximum Likelihood invariance properties.
The paper tackles the problem of training neural networks with mislabeled data by analyzing the effects of noise and showing that the Maximum Likelihood estimate from noisy data determines clean model parameters, leading to a method of adjusting decision thresholds based on noise levels or class priors.
We investigate the problem of machine learning with mislabeled training data. We try to make the effects of mislabeled training better understood through analysis of the basic model and equations that characterize the problem. This includes results about the ability of the noisy model to make the same decisions as the clean model and the effects of noise on model performance. In addition to providing better insights we also are able to show that the Maximum Likelihood (ML) estimate of the parameters of the noisy model determine those of the clean model. This property is obtained through the use of the ML invariance property and leads to an approach to developing a classifier when training has been mislabeled: namely train the classifier on noisy data and adjust the decision threshold based on the noise levels and/or class priors. We show how our approach to mislabeled training works with multi-layered perceptrons (MLPs).