Few-Shot Class-Incremental Learning with Prior Knowledge
This addresses incremental learning challenges for AI systems needing to adapt to new classes with limited data, though it appears incremental as it builds on prior methods by incorporating unlabeled data.
The paper tackles catastrophic forgetting and overfitting in few-shot class-incremental learning by proposing LwPK, which uses unlabeled data from incremental classes to enhance generalization, resulting in improved model resilience against forgetting.
To tackle the issues of catastrophic forgetting and overfitting in few-shot class-incremental learning (FSCIL), previous work has primarily concentrated on preserving the memory of old knowledge during the incremental phase. The role of pre-trained model in shaping the effectiveness of incremental learning is frequently underestimated in these studies. Therefore, to enhance the generalization ability of the pre-trained model, we propose Learning with Prior Knowledge (LwPK) by introducing nearly free prior knowledge from a few unlabeled data of subsequent incremental classes. We cluster unlabeled incremental class samples to produce pseudo-labels, then jointly train these with labeled base class samples, effectively allocating embedding space for both old and new class data. Experimental results indicate that LwPK effectively enhances the model resilience against catastrophic forgetting, with theoretical analysis based on empirical risk minimization and class distance measurement corroborating its operational principles. The source code of LwPK is publicly available at: \url{https://github.com/StevenJ308/LwPK}.