On Class Orderings for Incremental Learning
This addresses the problem of evaluating incremental learning methods more robustly for researchers, though it is incremental as it focuses on an overlooked aspect rather than a new method.
The paper investigates the impact of class orderings on incremental learning performance, finding that different orderings significantly affect results and method rankings.
The influence of class orderings in the evaluation of incremental learning has received very little attention. In this paper, we investigate the impact of class orderings for incrementally learned classifiers. We propose a method to compute various orderings for a dataset. The orderings are derived by simulated annealing optimization from the confusion matrix and reflect different incremental learning scenarios, including maximally and minimally confusing tasks. We evaluate a wide range of state-of-the-art incremental learning methods on the proposed orderings. Results show that orderings can have a significant impact on performance and the ranking of the methods.