Localizing Catastrophic Forgetting in Neural Networks
This work addresses catastrophic forgetting for researchers in continual learning, but it appears incremental as it focuses on localization rather than solving the problem.
The paper tackles the problem of catastrophic forgetting in neural networks by proposing a method to identify which parameters contribute most to forgetting, and applies it to analyze three continual learning scenarios.
Artificial neural networks (ANNs) suffer from catastrophic forgetting when trained on a sequence of tasks. While this phenomenon was studied in the past, there is only very limited recent research on this phenomenon. We propose a method for determining the contribution of individual parameters in an ANN to catastrophic forgetting. The method is used to analyze an ANNs response to three different continual learning scenarios.