Dynamical stability for dense patterns in discrete attractor neural networks
This addresses the challenge of reliable memory storage in biological and artificial neural networks, representing an incremental advance in theoretical understanding.
The paper tackled the problem of ensuring dynamical stability in neural networks storing multiple discrete attractors, which are models of biological memory, by deriving a theory that guarantees stability below a critical load distinct from classical capacity, with results highlighting benefits of threshold-linear activation and sparse-like patterns.
Neural networks storing multiple discrete attractors are canonical models of biological memory. Previously, the dynamical stability of such networks could only be guaranteed under highly restrictive conditions. Here, we derive a theory of the local stability of discrete fixed points in a broad class of networks with graded neural activities and in the presence of noise. By directly analyzing the bulk and outliers of the Jacobian spectrum, we show that all fixed points are stable below a critical load that is distinct from the classical \textit{critical capacity} and depends on the statistics of neural activities in the fixed points as well as the single-neuron activation function. Our analysis highlights the computational benefits of threshold-linear activation and sparse-like patterns.