Move-to-Data: A new Continual Learning approach with Deep CNNs, Application for image-class recognition
This incremental approach addresses computational efficiency for continual learning in image classification tasks.
The paper tackles the problem of adjusting a pre-trained model with new data for existing categories in continual learning, proposing a fast continual learning layer at the end of a neural network. Results on the CIFAR benchmark show similar performance to retraining but with drastically lower computational cost.
In many real-life tasks of application of supervised learning approaches, all the training data are not available at the same time. The examples are lifelong image classification or recognition of environmental objects during interaction of instrumented persons with their environment, enrichment of an online-database with more images. It is necessary to pre-train the model at a "training recording phase" and then adjust it to the new coming data. This is the task of incremental/continual learning approaches. Amongst different problems to be solved by these approaches such as introduction of new categories in the model, refining existing categories to sub-categories and extending trained classifiers over them, ... we focus on the problem of adjusting pre-trained model with new additional training data for existing categories. We propose a fast continual learning layer at the end of the neuronal network. Obtained results are illustrated on the opensource CIFAR benchmark dataset. The proposed scheme yields similar performances as retraining but with drastically lower computational cost.