Online Hyperparameter Optimization for Class-Incremental Learning
This work addresses the problem of improving CIL model performance for researchers and practitioners by providing an online hyperparameter optimization method that adapts to unknown settings, though it is incremental as it builds on existing CIL methods.
The paper tackles the challenge of adaptively optimizing the stability-plasticity tradeoff in class-incremental learning (CIL) across different data-receiving settings, achieving a 2.2 percentage point boost in average accuracy on ImageNet-Full compared to state-of-the-art methods.
Class-incremental learning (CIL) aims to train a classification model while the number of classes increases phase-by-phase. An inherent challenge of CIL is the stability-plasticity tradeoff, i.e., CIL models should keep stable to retain old knowledge and keep plastic to absorb new knowledge. However, none of the existing CIL models can achieve the optimal tradeoff in different data-receiving settings--where typically the training-from-half (TFH) setting needs more stability, but the training-from-scratch (TFS) needs more plasticity. To this end, we design an online learning method that can adaptively optimize the tradeoff without knowing the setting as a priori. Specifically, we first introduce the key hyperparameters that influence the trade-off, e.g., knowledge distillation (KD) loss weights, learning rates, and classifier types. Then, we formulate the hyperparameter optimization process as an online Markov Decision Process (MDP) problem and propose a specific algorithm to solve it. We apply local estimated rewards and a classic bandit algorithm Exp3 to address the issues when applying online MDP methods to the CIL protocol. Our method consistently improves top-performing CIL methods in both TFH and TFS settings, e.g., boosting the average accuracy of TFH and TFS by 2.2 percentage points on ImageNet-Full, compared to the state-of-the-art.