Self-Paced Imbalance Rectification for Class Incremental Learning
This work addresses the challenge of stabilizing incremental optimization in class-incremental learning for scenarios with limited and varying memory, which is an incremental improvement over existing methods.
The paper tackles the problem of class imbalance in exemplar-based class-incremental learning, where varying memory capacities cause ratio fluctuations between new and old samples, by proposing a self-paced imbalance rectification scheme that dynamically maintains balance during representation learning, resulting in stable incremental performance that significantly outperforms state-of-the-art methods on three benchmarks.
Exemplar-based class-incremental learning is to recognize new classes while not forgetting old ones, whose samples can only be saved in limited memory. The ratio fluctuation of new samples to old exemplars, which is caused by the variation of memory capacity at different environments, will bring challenges to stabilize the incremental optimization process. To address this problem, we propose a novel self-paced imbalance rectification scheme, which dynamically maintains the incremental balance during the representation learning phase. Specifically, our proposed scheme consists of a frequency compensation strategy that adjusts the logits margin between old and new classes with the corresponding number ratio to strengthen the expression ability of the old classes, and an inheritance transfer strategy to reduce the representation confusion by estimating the similarity of different classes in the old embedding space. Furthermore, a chronological attenuation mechanism is proposed to mitigate the repetitive optimization of the older classes at multiple step-wise increments. Extensive experiments on three benchmarks demonstrate stable incremental performance, significantly outperforming the state-of-the-art methods.