Latent Space based Memory Replay for Continual Learning in Artificial Neural Networks
This addresses the problem of continual learning for AI systems, though it appears incremental as it builds on existing memory replay methods.
The paper tackled catastrophic forgetting in artificial neural networks by using latent space based memory replay, achieving good performance on previous tasks while storing only a small percentage of the original data in a compressed form.
Memory replay may be key to learning in biological brains, which manage to learn new tasks continually without catastrophically interfering with previous knowledge. On the other hand, artificial neural networks suffer from catastrophic forgetting and tend to only perform well on tasks that they were recently trained on. In this work we explore the application of latent space based memory replay for classification using artificial neural networks. We are able to preserve good performance in previous tasks by storing only a small percentage of the original data in a compressed latent space version.