Recommendation of data-free class-incremental learning algorithms by simulating future data
This work addresses the practical deployment challenge of incremental learning for users by providing an automated algorithm selection tool, though it is incremental as it builds on existing algorithms and simulation techniques.
The paper tackles the problem of selecting appropriate class-incremental learning algorithms for user-defined settings by introducing a recommendation method that simulates future data streams using generative models, and it outperforms baselines with performance close to an oracle, achieving results like 95% accuracy in some settings.
Class-incremental learning deals with sequential data streams composed of batches of classes. Various algorithms have been proposed to address the challenging case where samples from past classes cannot be stored. However, selecting an appropriate algorithm for a user-defined setting is an open problem, as the relative performance of these algorithms depends on the incremental settings. To solve this problem, we introduce an algorithm recommendation method that simulates the future data stream. Given an initial set of classes, it leverages generative models to simulate future classes from the same visual domain. We evaluate recent algorithms on the simulated stream and recommend the one which performs best in the user-defined incremental setting. We illustrate the effectiveness of our method on three large datasets using six algorithms and six incremental settings. Our method outperforms competitive baselines, and performance is close to that of an oracle choosing the best algorithm in each setting. This work contributes to facilitate the practical deployment of incremental learning.