On-device Synaptic Memory Consolidation using Fowler-Nordheim Quantum-tunneling
This work addresses the challenge of energy-efficient continual learning for neuromorphic AI systems, representing an incremental advance by applying a novel device to a known bottleneck.
The paper tackles the problem of synaptic memory consolidation for continual learning in neuromorphic AI systems by proposing a Fowler-Nordheim quantum-tunneling device that stores synaptic weights and usage statistics on-device, resulting in outperforming a comparable EWC network on a small benchmark task with an energy footprint of femtojoules per update.
Synaptic memory consolidation has been heralded as one of the key mechanisms for supporting continual learning in neuromorphic Artificial Intelligence (AI) systems. Here we report that a Fowler-Nordheim (FN) quantum-tunneling device can implement synaptic memory consolidation similar to what can be achieved by algorithmic consolidation models like the cascade and the elastic weight consolidation (EWC) models. The proposed FN-synapse not only stores the synaptic weight but also stores the synapse's historical usage statistic on the device itself. We also show that the operation of the FN-synapse is near-optimal in terms of the synaptic lifetime and we demonstrate that a network comprising FN-synapses outperforms a comparable EWC network for a small benchmark continual learning task. With an energy footprint of femtojoules per synaptic update, we believe that the proposed FN-synapse provides an ultra-energy-efficient approach for implementing both synaptic memory consolidation and persistent learning.