TRGP: Trust Region Gradient Projection for Continual Learning
This addresses the problem of balancing performance on new and old tasks in continual learning for AI systems, though it appears incremental as it builds on existing constraint-based methods.
The paper tackles catastrophic forgetting in continual learning by proposing Trust Region Gradient Projection (TRGP), which uses a trust region to select related old tasks and scaled weight projection to reuse their weights, achieving significant improvement over state-of-the-art methods in experiments.
Catastrophic forgetting is one of the major challenges in continual learning. To address this issue, some existing methods put restrictive constraints on the optimization space of the new task for minimizing the interference to old tasks. However, this may lead to unsatisfactory performance for the new task, especially when the new task is strongly correlated with old tasks. To tackle this challenge, we propose Trust Region Gradient Projection (TRGP) for continual learning to facilitate the forward knowledge transfer based on an efficient characterization of task correlation. Particularly, we introduce a notion of `trust region' to select the most related old tasks for the new task in a layer-wise and single-shot manner, using the norm of gradient projection onto the subspace spanned by task inputs. Then, a scaled weight projection is proposed to cleverly reuse the frozen weights of the selected old tasks in the trust region through a layer-wise scaling matrix. By jointly optimizing the scaling matrices and the model, where the model is updated along the directions orthogonal to the subspaces of old tasks, TRGP can effectively prompt knowledge transfer without forgetting. Extensive experiments show that our approach achieves significant improvement over related state-of-the-art methods.