Local Information with Feedback Perturbation Suffices for Dictionary Learning in Neural Circuits
This addresses the problem of how neural circuits can learn sparse representations with local synaptic rules, which is incremental as it builds on existing sparse coding principles but introduces a novel mechanism.
The paper tackles the challenge of implementing dictionary learning under neural architectural constraints by introducing a spiking neural network that uses feedback synapses to convert non-local information into local signals, enabling it to solve the L1-minimizing dictionary learning problem as the first model to do so.
While the sparse coding principle can successfully model information processing in sensory neural systems, it remains unclear how learning can be accomplished under neural architectural constraints. Feasible learning rules must rely solely on synaptically local information in order to be implemented on spatially distributed neurons. We describe a neural network with spiking neurons that can address the aforementioned fundamental challenge and solve the L1-minimizing dictionary learning problem, representing the first model able to do so. Our major innovation is to introduce feedback synapses to create a pathway to turn the seemingly non-local information into local ones. The resulting network encodes the error signal needed for learning as the change of network steady states caused by feedback, and operates akin to the classical stochastic gradient descent method.