Self-Sustaining Representation Expansion for Non-Exemplar Class-Incremental Learning
This addresses the problem of catastrophic forgetting in class-incremental learning for AI systems when old data is unavailable, representing an incremental improvement over existing methods.
The paper tackles non-exemplar class-incremental learning, where old class samples cannot be saved, by proposing a self-sustaining representation expansion scheme with structure reorganization, main-branch distillation, and prototype selection. The method achieves significant incremental performance, outperforming state-of-the-art methods by margins of 3%, 3%, and 6% on three benchmarks.
Non-exemplar class-incremental learning is to recognize both the old and new classes when old class samples cannot be saved. It is a challenging task since representation optimization and feature retention can only be achieved under supervision from new classes. To address this problem, we propose a novel self-sustaining representation expansion scheme. Our scheme consists of a structure reorganization strategy that fuses main-branch expansion and side-branch updating to maintain the old features, and a main-branch distillation scheme to transfer the invariant knowledge. Furthermore, a prototype selection mechanism is proposed to enhance the discrimination between the old and new classes by selectively incorporating new samples into the distillation process. Extensive experiments on three benchmarks demonstrate significant incremental performance, outperforming the state-of-the-art methods by a margin of 3%, 3% and 6%, respectively.