EVCL: Elastic Variational Continual Learning with Weight Consolidation
This addresses the problem of forgetting in continual learning for AI systems, but it is incremental as it builds on prior methods.
The paper tackled catastrophic forgetting in continual learning by introducing EVCL, a hybrid model combining Variational Continual Learning and Elastic Weight Consolidation, which outperformed existing baselines on five discriminative tasks in domain-incremental and task-incremental scenarios.
Continual learning aims to allow models to learn new tasks without forgetting what has been learned before. This work introduces Elastic Variational Continual Learning with Weight Consolidation (EVCL), a novel hybrid model that integrates the variational posterior approximation mechanism of Variational Continual Learning (VCL) with the regularization-based parameter-protection strategy of Elastic Weight Consolidation (EWC). By combining the strengths of both methods, EVCL effectively mitigates catastrophic forgetting and enables better capture of dependencies between model parameters and task-specific data. Evaluated on five discriminative tasks, EVCL consistently outperforms existing baselines in both domain-incremental and task-incremental learning scenarios for deep discriminative models.