ReReLRP -- Remembering and Recognizing Tasks with LRP
This addresses the problem of catastrophic forgetting for continual learning applications, though it appears incremental as it builds on existing replay-free methods.
The paper tackles catastrophic forgetting in continual learning by proposing ReReLRP, which uses Layerwise Relevance Propagation to preserve information across tasks without replay mechanisms. The method achieves results comparable to replay-based approaches on various datasets while offering improved privacy, explainability, and memory efficiency.
Deep neural networks have revolutionized numerous research fields and applications. Despite their widespread success, a fundamental limitation known as catastrophic forgetting remains, where models fail to retain their ability to perform previously learned tasks after being trained on new ones. This limitation is particularly acute in certain continual learning scenarios, where models must integrate the knowledge from new domains with their existing capabilities. Traditional approaches to mitigate this problem typically rely on memory replay mechanisms, storing either original data samples, prototypes, or activation patterns. Although effective, these methods often introduce significant computational overhead, raise privacy concerns, and require the use of dedicated architectures. In this work we present ReReLRP (Remembering and Recognizing with LRP), a novel solution that leverages Layerwise Relevance Propagation (LRP) to preserve information across tasks. Our contribution provides increased privacy of existing replay-free methods while additionally offering built-in explainability, flexibility of model architecture and deployment, and a new mechanism to increase memory storage efficiency. We validate our approach on a wide variety of datasets, demonstrating results comparable with a well-known replay-based method in selected scenarios.