An Empirical Study of Incremental Learning in Neural Network with Noisy Training Set
This work addresses the problem of noisy training data in incremental learning for machine learning practitioners, but it is incremental as it builds on existing methods to analyze error effects.
The paper empirically studies how noise affects incremental learning in neural networks, finding that accuracy depends more on the location of errors than the percentage of errors, with results showing up to 20% variation in accuracy for the same error percentage across different error locations.
The notion of incremental learning is to train an ANN algorithm in stages, as and when newer training data arrives. Incremental learning is becoming widespread in recent times with the advent of deep learning. Noise in the training data reduces the accuracy of the algorithm. In this paper, we make an empirical study of the effect of noise in the training phase. We numerically show that the accuracy of the algorithm is dependent more on the location of the error than the percentage of error. Using Perceptron, Feed Forward Neural Network and Radial Basis Function Neural Network, we show that for the same percentage of error, the accuracy of the algorithm significantly varies with the location of error. Furthermore, our results show that the dependence of the accuracy with the location of error is independent of the algorithm. However, the slope of the degradation curve decreases with more sophisticated algorithms