Class-incremental Learning using a Sequence of Partial Implicitly Regularized Classifiers
This addresses the problem of forgetting in sequential learning for AI systems, but it appears incremental as it builds on experience replay and pretrained networks.
The paper tackles catastrophic forgetting in class-incremental learning by proposing a method that trains specialized classifiers instead of a single incremental classifier, showing a large performance improvement over state-of-the-art methods on the CIFAR100 dataset.
In class-incremental learning, the objective is to learn a number of classes sequentially without having access to the whole training data. However, due to a problem known as catastrophic forgetting, neural networks suffer substantial performance drop in such settings. The problem is often approached by experience replay, a method which stores a limited number of samples to be replayed in future steps to reduce forgetting of the learned classes. When using a pretrained network as a feature extractor, we show that instead of training a single classifier incrementally, it is better to train a number of specialized classifiers which do not interfere with each other yet can cooperatively predict a single class. Our experiments on CIFAR100 dataset show that the proposed method improves the performance over SOTA by a large margin.