Overcoming Catastrophic Forgetting by Generative Regularization
This addresses the problem of catastrophic forgetting for machine learning systems that need to learn sequentially, representing an incremental advancement in continual learning techniques.
The paper tackles catastrophic forgetting in continual learning by introducing generative regularization within a Bayesian inference framework, achieving performance improvements of over 15% on Fashion-MNIST and 10% on CUB datasets compared to baseline methods.
In this paper, we propose a new method to overcome catastrophic forgetting by adding generative regularization to Bayesian inference framework. Bayesian method provides a general framework for continual learning. We could further construct a generative regularization term for all given classification models by leveraging energy-based models and Langevin-dynamic sampling to enrich the features learned in each task. By combining discriminative and generative loss together, we empirically show that the proposed method outperforms state-of-the-art methods on a variety of tasks, avoiding catastrophic forgetting in continual learning. In particular, the proposed method outperforms baseline methods over 15% on the Fashion-MNIST dataset and 10% on the CUB dataset