Formalizing the Generalization-Forgetting Trade-off in Continual Learning
This work addresses the fundamental challenge of balancing forgetting and generalization in continual learning, which is crucial for AI systems that learn sequentially, though it appears incremental as it builds on existing game-theoretic formulations.
The authors tackled the trade-off between catastrophic forgetting and generalization in continual learning by formulating it as a two-player sequential game, showing theoretically that a stable balance point exists and introducing Balanced Continual Learning (BCL), which empirically performs comparably or better than state-of-the-art methods.
We formulate the continual learning (CL) problem via dynamic programming and model the trade-off between catastrophic forgetting and generalization as a two-player sequential game. In this approach, player 1 maximizes the cost due to lack of generalization whereas player 2 minimizes the cost due to catastrophic forgetting. We show theoretically that a balance point between the two players exists for each task and that this point is stable (once the balance is achieved, the two players stay at the balance point). Next, we introduce balanced continual learning (BCL), which is designed to attain balance between generalization and forgetting and empirically demonstrate that BCL is comparable to or better than the state of the art.