Lifelong Generative Learning via Knowledge Reconstruction
This addresses the problem of catastrophic forgetting in generative models for lifelong learning, though it appears incremental as it builds on existing VAE methods with specific improvements.
The paper tackles catastrophic forgetting in lifelong generative learning by developing an efficient variational autoencoder (VAE)-based model that uses knowledge reconstruction and feedback to retain historical knowledge, achieving results comparable to state-of-the-art on MNIST, FashionMNIST, and SVHN datasets.
Generative models often incur the catastrophic forgetting problem when they are used to sequentially learning multiple tasks, i.e., lifelong generative learning. Although there are some endeavors to tackle this problem, they suffer from high time-consumptions or error accumulation. In this work, we develop an efficient and effective lifelong generative model based on variational autoencoder (VAE). Unlike the generative adversarial network, VAE enjoys high efficiency in the training process, providing natural benefits with few resources. We deduce a lifelong generative model by expending the intrinsic reconstruction character of VAE to the historical knowledge retention. Further, we devise a feedback strategy about the reconstructed data to alleviate the error accumulation. Experiments on the lifelong generating tasks of MNIST, FashionMNIST, and SVHN verified the efficacy of our approach, where the results were comparable to SOTA.