Incremental Learning with Repetition via Pseudo-Feature Projection
This work addresses incremental learning for AI systems in practical settings with repetitive data, though it is incremental as it adapts existing approaches to new scenarios.
The paper tackled the problem of incremental learning in real-world scenarios where data streams contain repeated classes, proposing a new method (Horde) that dynamically adjusts an ensemble of feature extractors to exploit repetition. The method achieved competitive results without repetition and state-of-the-art performance with repetition, as demonstrated in their experiments.
Incremental Learning scenarios do not always represent real-world inference use-cases, which tend to have less strict task boundaries, and exhibit repetition of common classes and concepts in their continual data stream. To better represent these use-cases, new scenarios with partial repetition and mixing of tasks are proposed, where the repetition patterns are innate to the scenario and unknown to the strategy. We investigate how exemplar-free incremental learning strategies are affected by data repetition, and we adapt a series of state-of-the-art approaches to analyse and fairly compare them under both settings. Further, we also propose a novel method (Horde), able to dynamically adjust an ensemble of self-reliant feature extractors, and align them by exploiting class repetition. Our proposed exemplar-free method achieves competitive results in the classic scenario without repetition, and state-of-the-art performance in the one with repetition.