Watch Your Step: Optimal Retrieval for Continual Learning at Scale
This work addresses the selective retrieval bottleneck in continual learning for AI systems, but it is incremental as it builds on existing replay methods.
The paper tackled the problem of selective retrieval from replay buffers in continual learning, proposing a framework to evaluate and combine retrieval strategies and introducing methods to prevent duplicate replays, with experiments showing performance improvements on a large-scale object detection model fine-tuned on 15 datasets.
In continual learning, a model learns incrementally over time while minimizing interference between old and new tasks. One of the most widely used approaches in continual learning is referred to as replay. Replay methods support interleaved learning by storing past experiences in a replay buffer. Although there are methods for selectively constructing the buffer and reprocessing its contents, there is limited exploration of the problem of selectively retrieving samples from the buffer. Current solutions have been tested in limited settings and, more importantly, in isolation. Existing work has also not explored the impact of duplicate replays on performance. In this work, we propose a framework for evaluating selective retrieval strategies, categorized by simple, independent class- and sample-selective primitives. We evaluated several combinations of existing strategies for selective retrieval and present their performances. Furthermore, we propose a set of strategies to prevent duplicate replays and explore whether new samples with low loss values can be learned without replay. In an effort to match our problem setting to a realistic continual learning pipeline, we restrict our experiments to a setting involving a large, pre-trained, open vocabulary object detection model, which is fully fine-tuned on a sequence of 15 datasets.