Transfer Learning with Sparse Associative Memories
This work addresses the need for adaptable and efficient neural networks in resource-constrained environments, though it is incremental as it builds on existing associative memory concepts.
The paper tackles the problem of enabling incremental learning and real-time training on embedded devices for deep neural networks by introducing a novel output layer based on sparse associative memories, achieving more flexible and faster training with a slight accuracy decrease on ImageNet and domain-specific datasets.
In this paper, we introduce a novel layer designed to be used as the output of pre-trained neural networks in the context of classification. Based on Associative Memories, this layer can help design Deep Neural Networks which support incremental learning and that can be (partially) trained in real time on embedded devices. Experiments on the ImageNet dataset and other different domain specific datasets show that it is possible to design more flexible and faster-to-train Neural Networks at the cost of a slight decrease in accuracy.