Brain-inspired feature exaggeration in generative replay for continual learning
This addresses memory interference in continual learning models, which is crucial for developing reliable AI systems, though it appears incremental as it builds on existing generative replay techniques.
The paper tackles catastrophic forgetting in generative continual learning by applying a brain-inspired feature exaggeration method, achieving new state-of-the-art performance on early class classification in the CIFAR100 dataset.
The catastrophic forgetting of previously learnt classes is one of the main obstacles to the successful development of a reliable and accurate generative continual learning model. When learning new classes, the internal representation of previously learnt ones can often be overwritten, resulting in the model's "memory" of earlier classes being lost over time. Recent developments in neuroscience have uncovered a method through which the brain avoids its own form of memory interference. Applying a targeted exaggeration of the differences between features of similar, yet competing memories, the brain can more easily distinguish and recall them. In this paper, the application of such exaggeration, via the repulsion of replayed samples belonging to competing classes, is explored. Through the development of a 'reconstruction repulsion' loss, this paper presents a new state-of-the-art performance on the classification of early classes in the class-incremental learning dataset CIFAR100.