On the Discrimination and Consistency for Exemplar-Free Class Incremental Learning
This work addresses the problem of catastrophic forgetting in machine learning for scenarios where storing old data is not feasible, representing an incremental advance in task-specific methods for CIL.
The paper tackled the challenge of exemplar-free class incremental learning (EF-CIL) by proposing DCNet, which maps class representations into a hyperspherical space for better discrimination and consistency, achieving state-of-the-art performance with concrete improvements in accuracy.
Exemplar-free class incremental learning (EF-CIL) is a nontrivial task that requires continuously enriching model capability with new classes while maintaining previously learned knowledge without storing and replaying any old class exemplars. An emerging theory-guided framework for CIL trains task-specific models for a shared network, shifting the pressure of forgetting to task-id prediction. In EF-CIL, task-id prediction is more challenging due to the lack of inter-task interaction (e.g., replays of exemplars). To address this issue, we conduct a theoretical analysis of the importance and feasibility of preserving a discriminative and consistent feature space, upon which we propose a novel method termed DCNet. Concretely, it progressively maps class representations into a hyperspherical space, in which different classes are orthogonally distributed to achieve ample inter-class separation. Meanwhile, it also introduces compensatory training to adaptively adjust supervision intensity, thereby aligning the degree of intra-class aggregation. Extensive experiments and theoretical analysis verified the superiority of the proposed DCNet.