Learnable Expansion-and-Compression Network for Few-shot Class-Incremental Learning
This work addresses incremental learning with limited data, which is an important challenge for AI systems needing to adapt to new classes over time, though it appears incremental as it builds on existing FSCIL frameworks.
The paper tackles the problem of few-shot class-incremental learning (FSCIL) by proposing LEC-Net to address catastrophic forgetting and model over-fitting, achieving improvements of 5-7% over baselines and 5-6% over state-of-the-art methods on CUB/CIFAR-100 datasets.
Few-shot class-incremental learning (FSCIL), which targets at continuously expanding model's representation capacity under few supervisions, is an important yet challenging problem. On the one hand, when fitting new tasks (novel classes), features trained on old tasks (old classes) could significantly drift, causing catastrophic forgetting. On the other hand, training the large amount of model parameters with few-shot novel-class examples leads to model over-fitting. In this paper, we propose a learnable expansion-and-compression network (LEC-Net), with the aim to simultaneously solve catastrophic forgetting and model over-fitting problems in a unified framework. By tentatively expanding network nodes, LEC-Net enlarges the representation capacity of features, alleviating feature drift of old network from the perspective of model regularization. By compressing the expanded network nodes, LEC-Net purses minimal increase of model parameters, alleviating over-fitting of the expanded network from a perspective of compact representation. Experiments on the CUB/CIFAR-100 datasets show that LEC-Net improves the baseline by 5~7% while outperforms the state-of-the-art by 5~6%. LEC-Net also demonstrates the potential to be a general incremental learning approach with dynamic model expansion capability.