Exploring Example Influence in Continual Learning
This work addresses the challenge of catastrophic forgetting in continual learning for AI systems, offering an incremental improvement by optimizing example influence without requiring past data storage.
The paper tackled the problem of balancing stability and plasticity in continual learning by analyzing the influence of training examples, proposing a method to compute and fuse these influences for model updates and rehearsal storage optimization, achieving significant performance improvements over state-of-the-art methods on benchmark datasets.
Continual Learning (CL) sequentially learns new tasks like human beings, with the goal to achieve better Stability (S, remembering past tasks) and Plasticity (P, adapting to new tasks). Due to the fact that past training data is not available, it is valuable to explore the influence difference on S and P among training examples, which may improve the learning pattern towards better SP. Inspired by Influence Function (IF), we first study example influence via adding perturbation to example weight and computing the influence derivation. To avoid the storage and calculation burden of Hessian inverse in neural networks, we propose a simple yet effective MetaSP algorithm to simulate the two key steps in the computation of IF and obtain the S- and P-aware example influence. Moreover, we propose to fuse two kinds of example influence by solving a dual-objective optimization problem, and obtain a fused influence towards SP Pareto optimality. The fused influence can be used to control the update of model and optimize the storage of rehearsal. Empirical results show that our algorithm significantly outperforms state-of-the-art methods on both task- and class-incremental benchmark CL datasets.