Diffusion Model Meets Non-Exemplar Class-Incremental Learning and Beyond
This addresses the challenge of incremental learning without storing old data, which is crucial for privacy-sensitive applications, though it appears incremental as it builds on existing NECIL frameworks.
The paper tackles the problem of catastrophic forgetting in non-exemplar class-incremental learning (NECIL) by proposing DiffFR, a diffusion-based feature replay method that generates realistic features for replaying, resulting in a 3.0% average performance gain over state-of-the-art methods.
Non-exemplar class-incremental learning (NECIL) is to resist catastrophic forgetting without saving old class samples. Prior methodologies generally employ simple rules to generate features for replaying, suffering from large distribution gap between replayed features and real ones. To address the aforementioned issue, we propose a simple, yet effective \textbf{Diff}usion-based \textbf{F}eature \textbf{R}eplay (\textbf{DiffFR}) method for NECIL. First, to alleviate the limited representational capacity caused by fixing the feature extractor, we employ Siamese-based self-supervised learning for initial generalizable features. Second, we devise diffusion models to generate class-representative features highly similar to real features, which provides an effective way for exemplar-free knowledge memorization. Third, we introduce prototype calibration to direct the diffusion model's focus towards learning the distribution shapes of features, rather than the entire distribution. Extensive experiments on public datasets demonstrate significant performance gains of our DiffFR, outperforming the state-of-the-art NECIL methods by 3.0\% in average. The code will be made publicly available soon.