Classifying States of the Hopfield Network with Improved Accuracy, Generalization, and Interpretability
This work addresses the need for accurate and interpretable state classification in Hopfield networks, which is useful for tasks like ignoring spurious states during retrieval, but it is incremental as it builds on prior research with more complex models.
The paper tackles the problem of classifying states in Hopfield networks into categories like learned, spurious, and prototype states, finding that simple interpretable models such as feed-forward neural networks and support vector machines outperform traditional methods like the stability ratio, requiring little training data and generalizing well across different network configurations.
We extend the existing work on Hopfield network state classification, employing more complex models that remain interpretable, such as densely-connected feed-forward deep neural networks and support vector machines. The states of the Hopfield network can be grouped into several classes, including learned (those presented during training), spurious (stable states that were not learned), and prototype (stable states that were not learned but are representative for a subset of learned states). It is often useful to determine to what class a given state belongs to; for example to ignore spurious states when retrieving from the network. Previous research has approached the state classification task with simple linear methods, most notably the stability ratio. We deepen the research on classifying states from prototype-regime Hopfield networks, investigating how varying the factors strengthening prototypes influences the state classification task. We study the generalizability of different classification models when trained on states derived from different prototype tasks -- for example, can a network trained on a Hopfield network with 10 prototypes classify states from a network with 20 prototypes? We find that simple models often outperform the stability ratio while remaining interpretable. These models require surprisingly little training data and generalize exceptionally well to states generated by a range of Hopfield networks, even those that were trained on exceedingly different datasets.