Relational Experience Replay: Continual Learning by Adaptively Tuning Task-wise Relationship
This work addresses the challenge of balancing stability and plasticity in continual learning for AI systems, though it appears incremental as it builds on rehearsal-based methods.
The paper tackled the problem of catastrophic forgetting in continual learning by proposing Relational Experience Replay (RER), which adaptively tunes task-wise relationships and sample importance, resulting in improved performance over baselines and state-of-the-art methods on datasets like CIFAR-10, CIFAR-100, and Tiny ImageNet.
Continual learning is a promising machine learning paradigm to learn new tasks while retaining previously learned knowledge over streaming training data. Till now, rehearsal-based methods, keeping a small part of data from old tasks as a memory buffer, have shown good performance in mitigating catastrophic forgetting for previously learned knowledge. However, most of these methods typically treat each new task equally, which may not adequately consider the relationship or similarity between old and new tasks. Furthermore, these methods commonly neglect sample importance in the continual training process and result in sub-optimal performance on certain tasks. To address this challenging problem, we propose Relational Experience Replay (RER), a bi-level learning framework, to adaptively tune task-wise relationships and sample importance within each task to achieve a better `stability' and `plasticity' trade-off. As such, the proposed method is capable of accumulating new knowledge while consolidating previously learned old knowledge during continual learning. Extensive experiments conducted on three publicly available datasets (i.e., CIFAR-10, CIFAR-100, and Tiny ImageNet) show that the proposed method can consistently improve the performance of all baselines and surpass current state-of-the-art methods.