Balanced Supervised Contrastive Learning for Few-Shot Class-Incremental Learning
This work addresses the problem of catastrophic forgetting in incremental learning for AI systems, representing an incremental improvement with specific gains in few-shot scenarios.
The paper tackles the challenge of balancing underfitting to new tasks and forgetting past tasks in few-shot class-incremental learning by developing a learning scheme with a balanced supervised contrastive loss and unified classifier initialization, achieving significant improvements over state-of-the-art methods on CUB200, CIFAR100, and miniImagenet datasets.
Few-shot class-incremental learning (FSCIL) presents the primary challenge of balancing underfitting to a new session's task and forgetting the tasks from previous sessions. To address this challenge, we develop a simple yet powerful learning scheme that integrates effective methods for each core component of the FSCIL network, including the feature extractor, base session classifiers, and incremental session classifiers. In feature extractor training, our goal is to obtain balanced generic representations that benefit both current viewable and unseen or past classes. To achieve this, we propose a balanced supervised contrastive loss that effectively balances these two objectives. In terms of classifiers, we analyze and emphasize the importance of unifying initialization methods for both the base and incremental session classifiers. Our method demonstrates outstanding ability for new task learning and preventing forgetting on CUB200, CIFAR100, and miniImagenet datasets, with significant improvements over previous state-of-the-art methods across diverse metrics. We conduct experiments to analyze the significance and rationale behind our approach and visualize the effectiveness of our representations on new tasks. Furthermore, we conduct diverse ablation studies to analyze the effects of each module.