Continual Learning of Numerous Tasks from Long-tail Distributions
This addresses a practical limitation in continual learning for real-world applications where tasks vary widely in size, though it's an incremental improvement to existing methods.
The paper tackles the problem of continual learning with numerous tasks from long-tail distributions, showing that existing algorithms perform poorly in such realistic settings. The authors propose reusing Adam optimizer states via weighted averaging of second moments, which reduces forgetting with minimal computational overhead and improves existing algorithms, particularly achieving up to 15% accuracy gains in long-tail sequences.
Continual learning, an important aspect of artificial intelligence and machine learning research, focuses on developing models that learn and adapt to new tasks while retaining previously acquired knowledge. Existing continual learning algorithms usually involve a small number of tasks with uniform sizes and may not accurately represent real-world learning scenarios. In this paper, we investigate the performance of continual learning algorithms with a large number of tasks drawn from a task distribution that is long-tail in terms of task sizes. We design one synthetic dataset and two real-world continual learning datasets to evaluate the performance of existing algorithms in such a setting. Moreover, we study an overlooked factor in continual learning, the optimizer states, e.g. first and second moments in the Adam optimizer, and investigate how it can be used to improve continual learning performance. We propose a method that reuses the optimizer states in Adam by maintaining a weighted average of the second moments from previous tasks. We demonstrate that our method, compatible with most existing continual learning algorithms, effectively reduces forgetting with only a small amount of additional computational or memory costs, and provides further improvements on existing continual learning algorithms, particularly in a long-tail task sequence.