Benchmarking local Hebbian learning rules for memory storage and prototype extraction
For researchers in neural networks and computational neuroscience, this provides a comparative evaluation of Hebbian learning rules for memory and prototype extraction tasks.
This paper benchmarks seven Hebbian learning rules for associative memory in recurrent networks, finding that Bayesian-Hebbian rules achieve the highest capacity for pattern storage and prototype extraction, while the additive Hebb rule performs worst.
Associative memory or content-addressable memory is an important component function in computer science and information processing, and at the same time a key concept in cognitive and computational brain science. Many different neural network architectures and learning rules have been proposed to model the brain's associative memory while investigating key component functions like figure-ground segmentation, perceptual reconstruction and rivalry. A less investigated but equally important capability of associative memory is prototype extraction where the training set comprises distorted prototype instances and the task is to recall the correct generating prototype given a new distorted instance. In this paper we benchmark associative memory function of seven different Hebbian learning rules employed in non-modular and modular recurrent networks with winner-take-all dynamics operating on moderately sparse binary patterns. We measure pattern storage and weight information capacity, prototype extraction capabilities, and sensitivity to correlations in data. The original additive Hebb rule comes out with worst capacity, covariance learning proves to be robust but with moderate capacity, and the Bayesian-Hebbian learning rules show highest capacity in almost all different conditions tested.