DELTA: Decoupling Long-Tailed Online Continual Learning
This addresses the challenge of enabling AI models to learn efficiently from real-world, imbalanced data streams in continual learning settings, which is incremental as it builds on existing OCL methods.
The paper tackles the problem of Long-Tailed Online Continual Learning (LTOCL), where models must learn from sequentially arriving class-imbalanced data streams without forgetting previous knowledge, and presents DELTA, a decoupled learning approach that improves incremental learning capacity and surpasses existing OCL methods.
A significant challenge in achieving ubiquitous Artificial Intelligence is the limited ability of models to rapidly learn new information in real-world scenarios where data follows long-tailed distributions, all while avoiding forgetting previously acquired knowledge. In this work, we study the under-explored problem of Long-Tailed Online Continual Learning (LTOCL), which aims to learn new tasks from sequentially arriving class-imbalanced data streams. Each data is observed only once for training without knowing the task data distribution. We present DELTA, a decoupled learning approach designed to enhance learning representations and address the substantial imbalance in LTOCL. We enhance the learning process by adapting supervised contrastive learning to attract similar samples and repel dissimilar (out-of-class) samples. Further, by balancing gradients during training using an equalization loss, DELTA significantly enhances learning outcomes and successfully mitigates catastrophic forgetting. Through extensive evaluation, we demonstrate that DELTA improves the capacity for incremental learning, surpassing existing OCL methods. Our results suggest considerable promise for applying OCL in real-world applications.