Exemplar-Free Class Incremental Learning via Incremental Representation
This addresses the problem of continuous learning without storing old data for machine learning systems, though it appears incremental as it builds on existing efCIL methods with a simpler approach.
The paper tackles exemplar-free class incremental learning (efCIL) by proposing a simple Incremental Representation (IR) framework that avoids constructing old pseudo-features, using dataset augmentation and an L2 space maintenance loss to prevent forgetting, achieving comparable performance on CIFAR100, TinyImageNet, and ImageNetSubset datasets.
Exemplar-Free Class Incremental Learning (efCIL) aims to continuously incorporate the knowledge from new classes while retaining previously learned information, without storing any old-class exemplars (i.e., samples). For this purpose, various efCIL methods have been proposed over the past few years, generally with elaborately constructed old pseudo-features, increasing the difficulty of model development and interpretation. In contrast, we propose a \textbf{simple Incremental Representation (IR) framework} for efCIL without constructing old pseudo-features. IR utilizes dataset augmentation to cover a suitable feature space and prevents the model from forgetting by using a single L2 space maintenance loss. We discard the transient classifier trained on each one of the sequence tasks and instead replace it with a 1-near-neighbor classifier for inference, ensuring the representation is incrementally updated during CIL. Extensive experiments demonstrate that our proposed IR achieves comparable performance while significantly preventing the model from forgetting on CIFAR100, TinyImageNet, and ImageNetSubset datasets.