Adaptive Discovering and Merging for Incremental Novel Class Discovery
This addresses the challenge of continuous learning in AI systems by enabling adaptive discovery and integration of new knowledge without disrupting existing capabilities, though it appears incremental as it builds on prior class-incremental learning methods.
The paper tackles the problem of discovering novel classes from unlabeled data in lifelong learning while preventing catastrophic forgetting, and introduces the Adaptive Discovering and Merging (ADM) method, which significantly outperforms existing approaches in class-incremental Novel Class Discovery and also benefits class-incremental Learning by reducing forgetting.
One important desideratum of lifelong learning aims to discover novel classes from unlabelled data in a continuous manner. The central challenge is twofold: discovering and learning novel classes while mitigating the issue of catastrophic forgetting of established knowledge. To this end, we introduce a new paradigm called Adaptive Discovering and Merging (ADM) to discover novel categories adaptively in the incremental stage and integrate novel knowledge into the model without affecting the original knowledge. To discover novel classes adaptively, we decouple representation learning and novel class discovery, and use Triple Comparison (TC) and Probability Regularization (PR) to constrain the probability discrepancy and diversity for adaptive category assignment. To merge the learned novel knowledge adaptively, we propose a hybrid structure with base and novel branches named Adaptive Model Merging (AMM), which reduces the interference of the novel branch on the old classes to preserve the previous knowledge, and merges the novel branch to the base model without performance loss and parameter growth. Extensive experiments on several datasets show that ADM significantly outperforms existing class-incremental Novel Class Discovery (class-iNCD) approaches. Moreover, our AMM also benefits the class-incremental Learning (class-IL) task by alleviating the catastrophic forgetting problem.