Anti-Retroactive Interference for Lifelong Learning
This addresses the problem of forgetting in machine learning models for lifelong learning, offering a novel approach with potential broad impact, though it appears incremental as it builds on existing meta-learning and cognitive science concepts.
The paper tackles catastrophic forgetting in lifelong learning by introducing a meta-learning paradigm that disrupts background distributions to enhance feature extraction and adaptively fuses incremental knowledge based on similarity, achieving improved performance on MNIST, CIFAR100, CUB200, and ImageNet100 datasets.
Humans can continuously learn new knowledge. However, machine learning models suffer from drastic dropping in performance on previous tasks after learning new tasks. Cognitive science points out that the competition of similar knowledge is an important cause of forgetting. In this paper, we design a paradigm for lifelong learning based on meta-learning and associative mechanism of the brain. It tackles the problem from two aspects: extracting knowledge and memorizing knowledge. First, we disrupt the sample's background distribution through a background attack, which strengthens the model to extract the key features of each task. Second, according to the similarity between incremental knowledge and base knowledge, we design an adaptive fusion of incremental knowledge, which helps the model allocate capacity to the knowledge of different difficulties. It is theoretically analyzed that the proposed learning paradigm can make the models of different tasks converge to the same optimum. The proposed method is validated on the MNIST, CIFAR100, CUB200 and ImageNet100 datasets.