Grow, Assess, Compress: Adaptive Backbone Scaling for Memory-Efficient Class Incremental Learning
This addresses memory inefficiency in incremental learning for AI systems, offering a novel solution to prevent parameter explosion while maintaining performance.
The paper tackles the challenge of balancing plasticity and stability in Class Incremental Learning by proposing a dynamic scaling framework that adaptively manages model capacity, achieving state-of-the-art performance and reducing memory footprint by up to 73% compared to expansion-based methods.
Class Incremental Learning (CIL) poses a fundamental challenge: maintaining a balance between the plasticity required to learn new tasks and the stability needed to prevent catastrophic forgetting. While expansion-based methods effectively mitigate forgetting by adding task-specific parameters, they suffer from uncontrolled architectural growth and memory overhead. In this paper, we propose a novel dynamic scaling framework that adaptively manages model capacity through a cyclic "GRow, Assess, ComprEss" (GRACE) strategy. Crucially, we supplement backbone expansion with a novel saturation assessment phase that evaluates the utilization of the model's capacity. This assessment allows the framework to make informed decisions to either expand the architecture or compress the backbones into a streamlined representation, preventing parameter explosion. Experimental results demonstrate that our approach achieves state-of-the-art performance across multiple CIL benchmarks, while reducing memory footprint by up to a 73% compared to purely expansionist models.