Continual Learning with Columnar Spiking Neural Networks
This addresses the problem of catastrophic forgetting in artificial neural networks for continual learning applications, representing an incremental improvement with biologically inspired methods.
The study tackled catastrophic forgetting in continual learning by proposing columnar-organized spiking neural networks with local learning rules, achieving 92% accuracy on each of ten sequential tasks and only 4% performance degradation on the first task after subsequent training.
Continual learning is a key feature of biological neural systems, but artificial neural networks often suffer from catastrophic forgetting. Instead of backpropagation, biologically plausible learning algorithms may enable stable continual learning. This study proposes columnar-organized spiking neural networks (SNNs) with local learning rules for continual learning and catastrophic forgetting. Using CoLaNET (Columnar Layered Network), we show that its microcolumns adapt most efficiently to new tasks when they lack shared structure with prior learning. We demonstrate how CoLaNET hyperparameters govern the trade-off between retaining old knowledge (stability) and acquiring new information (plasticity). We evaluate CoLaNET on two benchmarks: Permuted MNIST (ten sequential pixel-permuted tasks) and a two-task MNIST/EMNIST setup. Our model learns ten sequential tasks effectively, maintaining 92% accuracy on each. It shows low forgetting, with only 4% performance degradation on the first task after training on nine subsequent tasks.