Beyond Fixed Points: Superpolynomial Capacity of Asymmetric Hopfield Networks
For researchers in neural networks and memory models, this work shows that simple asymmetric Hopfield networks can robustly store exponentially many long sequences, challenging previous limitations and suggesting that biological and artificial systems can achieve robust sequence representation without complex nonlinearities.
The authors present a construction for asymmetric Hopfield networks with binary neurons and synchronous updates that achieves superpolynomial capacity, storing exp(Ω(n/(log n)^2)) distinct limit-cycle attractors each with period exp(Ω(√n/log n)), robust to noise with flip probability up to 1/2 - o(1). This is the first demonstration of such high capacity for sequence memory in these networks.
Classical Hopfield networks are limited to static patterns due to symmetric weights, whereas asymmetric networks can encode temporal sequences via limit-cycle attractors. Achieving high-capacity storage of long sequences in classical synchronous asymmetric networks, however, has remained a challenge. We present a simple and robust construction within the classical asymmetric Hopfield model with binary neurons and synchronous updates, that allows $n$ neurons to support $\exp\!\big(Ω(n/(\log n)^2)\big)$ distinct limit-cycle attractors, each with period $\exp\!\big(Ω(\sqrt n/\log n)\big)$ and robust to random noise with flip probability up to $\frac12-o(1)$, yielding superpolynomial capacity in both the number and length of stored sequences. This is the first demonstration of such capacity for asymmetric Hopfield networks, which we obtain by combining results from combinatorics, number theory and the analysis of opinion dynamics. Our findings show that synchronous asymmetric Hopfield networks possess a sequence-memory capacity which is larger and more robust than previously recognized, demonstrating that, in both biological and artificial neural systems, robust sequence representation can be achieved through coarse architectural motifs rather than complex nonlinearities.