Pseudo Rehearsal using non photo-realistic images
This addresses the problem of catastrophic forgetting for machine learning practitioners by offering a more efficient incremental improvement over existing pseudo-rehearsal methods.
The paper tackles catastrophic forgetting in deep neural networks by showing that pseudo-rehearsal with non-photo-realistic synthetic images can effectively retain previous tasks, achieving good retention while significantly reducing computational and memory resource consumption.
Deep Neural networks forget previously learnt tasks when they are faced with learning new tasks. This is called catastrophic forgetting. Rehearsing the neural network with the training data of the previous task can protect the network from catastrophic forgetting. Since rehearsing requires the storage of entire previous data, Pseudo rehearsal was proposed, where samples belonging to the previous data are generated synthetically for rehearsal. In an image classification setting, while current techniques try to generate synthetic data that is photo-realistic, we demonstrated that Neural networks can be rehearsed on data that is not photo-realistic and still achieve good retention of the previous task. We also demonstrated that forgoing the constraint of having photo realism in the generated data can result in a significant reduction in the consumption of computational and memory resources for pseudo rehearsal.