Energy-convergence trade off for the training of neural networks on bio-inspired hardware
This work addresses the need for ultra-low power AI processing on wearable and implantable devices, though it is incremental as it builds on existing hardware-aware methods to improve energy efficiency.
The study tackled the challenge of balancing performance and energy efficiency in training neural networks on bio-inspired hardware by investigating ferroelectric synaptic devices, finding that shorter programming pulses reduce total energy without sacrificing accuracy but require more training epochs, and proposing a 'symmetry point shifting' technique to restore accuracy diminished by asymmetric updates.
The increasing deployment of wearable sensors and implantable devices is shifting AI processing demands to the extreme edge, necessitating ultra-low power for continuous operation. Inspired by the brain, emerging memristive devices promise to accelerate neural network training by eliminating costly data transfers between compute and memory. Though, balancing performance and energy efficiency remains a challenge. We investigate ferroelectric synaptic devices based on HfO2/ZrO2 superlattices and feed their experimentally measured weight updates into hardware-aware neural network simulations. Across pulse widths from 20 ns to 0.2 ms, shorter pulses lower per-update energy but require more training epochs while still reducing total energy without sacrificing accuracy. Classification accuracy using plain stochastic gradient descent (SGD) is diminished compared to mixed-precision SGD. We analyze the causes and propose a ``symmetry point shifting'' technique, addressing asymmetric updates and restoring accuracy. These results highlight a trade-off among accuracy, convergence speed, and energy use, showing that short-pulse programming with tailored training significantly enhances on-chip learning efficiency.