Kernel Ridge Regression for Efficient Learning of High-Capacity Hopfield Networks
This work addresses the problem of building high-performance associative memories for applications requiring efficient learning, though it is incremental as it adapts an existing kernel method to a specific domain.
The paper tackles the limited storage capacity of Hopfield networks by proposing Kernel Ridge Regression (KRR) as an efficient kernel-based learning method, achieving state-of-the-art storage capacity (up to a load of 1.5) and noise robustness comparable to Kernel Logistic Regression (KLR) while drastically reducing training time, being orders of magnitude faster than Linear Logistic Regression (LLR) and significantly faster than KLR.
Hopfield networks using Hebbian learning suffer from limited storage capacity. While supervised methods like Linear Logistic Regression (LLR) offer some improvement, kernel methods like Kernel Logistic Regression (KLR) significantly enhance storage capacity and noise robustness. However, KLR requires computationally expensive iterative learning. We propose Kernel Ridge Regression (KRR) as an efficient kernel-based alternative for learning high-capacity Hopfield networks. KRR utilizes the kernel trick and predicts bipolar states via regression, crucially offering a non-iterative, closed-form solution for learning dual variables. We evaluate KRR and compare its performance against Hebbian, LLR, and KLR. Our results demonstrate that KRR achieves state-of-the-art storage capacity (reaching a storage load of 1.5) and noise robustness, comparable to KLR. Crucially, KRR drastically reduces training time, being orders of magnitude faster than LLR and significantly faster than KLR, especially at higher storage loads. This establishes KRR as a potent and highly efficient method for building high-performance associative memories, providing comparable performance to KLR with substantial training speed advantages. This work provides the first empirical comparison between KRR and KLR in the context of Hopfield network learning.