EKPC: Elastic Knowledge Preservation and Compensation for Class-Incremental Learning
This addresses the challenge of enabling AI models to learn new classes over time without forgetting old ones, which is crucial for real-world applications, though it appears incremental as it builds on existing parameter-efficient fine-tuning approaches.
The paper tackles the problem of class-incremental learning (CIL) by proposing the EKPC method, which integrates importance-aware parameter regularization and trainable semantic drift compensation to preserve knowledge and reduce forgetting, achieving superior performance on five benchmarks compared to state-of-the-art methods.
Class-Incremental Learning (CIL) aims to enable AI models to continuously learn from sequentially arriving data of different classes over time while retaining previously acquired knowledge. Recently, Parameter-Efficient Fine-Tuning (PEFT) methods, like prompt pool-based approaches and adapter tuning, have shown great attraction in CIL. However, these methods either introduce additional parameters that increase memory usage, or rely on rigid regularization techniques which reduce forgetting but compromise model flexibility. To overcome these limitations, we propose the Elastic Knowledge Preservation and Compensation (EKPC) method, integrating Importance-aware Parameter Regularization (IPR) and Trainable Semantic Drift Compensation (TSDC) for CIL. Specifically, the IPR method assesses the sensitivity of network parameters to prior tasks using a novel parameter-importance algorithm. It then selectively constrains updates within the shared adapter according to these importance values, thereby preserving previously acquired knowledge while maintaining the model's flexibility. However, it still exhibits slight semantic differences in previous knowledge to accommodate new incremental tasks, leading to decision boundaries confusion in classifier. To eliminate this confusion, TSDC trains a unified classifier by compensating prototypes with trainable semantic drift. Extensive experiments on five CIL benchmarks demonstrate the effectiveness of the proposed method, showing superior performances to existing state-of-the-art methods.