Lazy learning: a biologically-inspired plasticity rule for fast and energy efficient synaptic plasticity
This method addresses the problem of computational and energy inefficiency in neural network training for large datasets, offering a practical improvement.
The paper tackles the inefficiency of updating neural network parameters on every training sample by introducing lazy learning, a biologically-inspired rule that updates only on incorrect samples, achieving 99.2% test accuracy on Extended MNIST with a single-layer MLP and a 7.6x speedup compared to backpropagation.
When training neural networks for classification tasks with backpropagation, parameters are updated on every trial, even if the sample is classified correctly. In contrast, humans concentrate their learning effort on errors. Inspired by human learning, we introduce lazy learning, which only learns on incorrect samples. Lazy learning can be implemented in a few lines of code and requires no hyperparameter tuning. Lazy learning achieves state-of-the-art performance and is particularly suited when datasets are large. For instance, it reaches 99.2% test accuracy on Extended MNIST using a single-layer MLP, and does so 7.6x faster than a matched backprop network