Rotation Augmented Distillation for Exemplar-Free Class Incremental Learning with Detailed Analysis
This work addresses catastrophic forgetting in deep neural networks for incremental learning without saving old data, which is crucial for applications with privacy or storage constraints, though it is incremental in method.
The paper tackles the problem of exemplar-free class incremental learning (CIL) by proposing Rotation Augmented Distillation (RAD), which achieves top-tier performance by balancing plasticity and stability, as demonstrated through detailed analysis and comparisons in challenging settings.
Class incremental learning (CIL) aims to recognize both the old and new classes along the increment tasks. Deep neural networks in CIL suffer from catastrophic forgetting and some approaches rely on saving exemplars from previous tasks, known as the exemplar-based setting, to alleviate this problem. On the contrary, this paper focuses on the Exemplar-Free setting with no old class sample preserved. Balancing the plasticity and stability in deep feature learning with only supervision from new classes is more challenging. Most existing Exemplar-Free CIL methods report the overall performance only and lack further analysis. In this work, different methods are examined with complementary metrics in greater detail. Moreover, we propose a simple CIL method, Rotation Augmented Distillation (RAD), which achieves one of the top-tier performances under the Exemplar-Free setting. Detailed analysis shows our RAD benefits from the superior balance between plasticity and stability. Finally, more challenging exemplar-free settings with fewer initial classes are undertaken for further demonstrations and comparisons among the state-of-the-art methods.