Adaptive Weighted Parameter Fusion with CLIP for Class-Incremental Learning
This addresses the problem of forgetting old classes when learning new ones in incremental learning, but it is incremental as it builds on existing methods like CLIP.
The paper tackles catastrophic forgetting in class-incremental learning by proposing an adaptive weighted parameter fusion method with CLIP, which balances retaining old knowledge and accommodating new information, achieving superior results on traditional benchmarks.
Class-incremental Learning (CIL) enables the model to incrementally absorb knowledge from new classes and build a generic classifier across all previously encountered classes. When the model optimizes with new classes, the knowledge of previous classes is inevitably erased, leading to catastrophic forgetting. Addressing this challenge requires making a trade-off between retaining old knowledge and accommodating new information. However, this balancing process often requires sacrificing some information, which can lead to a partial loss in the model's ability to discriminate between classes. To tackle this issue, we design the adaptive weighted parameter fusion with Contrastive Language-Image Pre-training (CLIP), which not only takes into account the variability of the data distribution of different tasks, but also retains all the effective information of the parameter matrix to the greatest extent. In addition, we introduce a balance factor that can balance the data distribution alignment and distinguishability of adjacent tasks. Experimental results on several traditional benchmarks validate the superiority of the proposed method.