Elastic Feature Consolidation for Cold Start Exemplar-Free Incremental Learning
This addresses a challenging incremental learning scenario for AI systems that need to adapt to new tasks without storing old data, but it is incremental as it builds on existing EFCIL methods.
The paper tackles the problem of Cold Start Exemplar-Free Class Incremental Learning, where insufficient initial data leads to feature drift, by proposing Elastic Feature Consolidation to regularize drift and use prototypes, resulting in significant outperformance of state-of-the-art methods on datasets like CIFAR-100 and ImageNet-1K.
Exemplar-Free Class Incremental Learning (EFCIL) aims to learn from a sequence of tasks without having access to previous task data. In this paper, we consider the challenging Cold Start scenario in which insufficient data is available in the first task to learn a high-quality backbone. This is especially challenging for EFCIL since it requires high plasticity, which results in feature drift which is difficult to compensate for in the exemplar-free setting. To address this problem, we propose a simple and effective approach that consolidates feature representations by regularizing drift in directions highly relevant to previous tasks and employs prototypes to reduce task-recency bias. Our method, called Elastic Feature Consolidation (EFC), exploits a tractable second-order approximation of feature drift based on an Empirical Feature Matrix (EFM). The EFM induces a pseudo-metric in feature space which we use to regularize feature drift in important directions and to update Gaussian prototypes used in a novel asymmetric cross entropy loss which effectively balances prototype rehearsal with data from new tasks. Experimental results on CIFAR-100, Tiny-ImageNet, ImageNet-Subset and ImageNet-1K demonstrate that Elastic Feature Consolidation is better able to learn new tasks by maintaining model plasticity and significantly outperform the state-of-the-art.