CULT: Continual Unsupervised Learning with Typicality-Based Environment Detection
This addresses the challenge of maintaining learned representations over time in unsupervised learning for AI systems, but it is incremental as it builds on existing methods like generative replay.
The paper tackles the problem of catastrophic forgetting in continual unsupervised representation learning by introducing CULT, which uses a typicality metric in VAE latent space to detect distributional shifts, and it significantly outperforms baseline approaches.
We introduce CULT (Continual Unsupervised Representation Learning with Typicality-Based Environment Detection), a new algorithm for continual unsupervised learning with variational auto-encoders. CULT uses a simple typicality metric in the latent space of a VAE to detect distributional shifts in the environment, which is used in conjunction with generative replay and an auxiliary environmental classifier to limit catastrophic forgetting in unsupervised representation learning. In our experiments, CULT significantly outperforms baseline continual unsupervised learning approaches. Code for this paper can be found here: https://github.com/oliveradk/cult