Continual Learning with Neuron Activation Importance
This work addresses the challenge of stable online learning for AI systems that need to adapt to new tasks without forgetting old ones, representing an incremental improvement over existing methods.
The paper tackles the problem of catastrophic forgetting in continual learning by proposing a neuron activation importance-based regularization method, achieving improved classification accuracy and robustness to task order changes in experiments on benchmark datasets.
Continual learning is a concept of online learning with multiple sequential tasks. One of the critical barriers of continual learning is that a network should learn a new task keeping the knowledge of old tasks without access to any data of the old tasks. In this paper, we propose a neuron activation importance-based regularization method for stable continual learning regardless of the order of tasks. We conduct comprehensive experiments on existing benchmark data sets to evaluate not just the stability and plasticity of our method with improved classification accuracy also the robustness of the performance along the changes of task order.