Can Synthetic Images Conquer Forgetting? Beyond Unexplored Doubts in Few-Shot Class-Incremental Learning
This addresses the problem of catastrophic forgetting in incremental learning for AI systems, but it is incremental as it builds on existing diffusion models and FSCIL methods.
The paper tackles the challenge of few-shot class-incremental learning (FSCIL) by proposing Diffusion-FSCIL, which uses a frozen text-to-image diffusion model as a backbone to reduce catastrophic forgetting and learn new classes with limited data, achieving state-of-the-art results on benchmarks like CUB-200, miniImageNet, and CIFAR-100.
Few-shot class-incremental learning (FSCIL) is challenging due to extremely limited training data; while aiming to reduce catastrophic forgetting and learn new information. We propose Diffusion-FSCIL, a novel approach that employs a text-to-image diffusion model as a frozen backbone. Our conjecture is that FSCIL can be tackled using a large generative model's capabilities benefiting from 1) generation ability via large-scale pre-training; 2) multi-scale representation; 3) representational flexibility through the text encoder. To maximize the representation capability, we propose to extract multiple complementary diffusion features to play roles as latent replay with slight support from feature distillation for preventing generative biases. Our framework realizes efficiency through 1) using a frozen backbone; 2) minimal trainable components; 3) batch processing of multiple feature extractions. Extensive experiments on CUB-200, \emph{mini}ImageNet, and CIFAR-100 show that Diffusion-FSCIL surpasses state-of-the-art methods, preserving performance on previously learned classes and adapting effectively to new ones.