Unsupervised prototype learning in an associative-memory network
This work addresses unsupervised representation learning for AI systems, offering a method to abstract data into prototypes, but it appears incremental as it builds on existing Hopfield and Boltzmann machine frameworks.
The paper tackled unsupervised learning in a generalized Hopfield associative-memory network by proving its equivalence to a semi-restricted Boltzmann machine and demonstrating that it learns prototypes as internal representations of input data, with a spectral method proposed to extract concise idealized prototypes.
Unsupervised learning in a generalized Hopfield associative-memory network is investigated in this work. First, we prove that the (generalized) Hopfield model is equivalent to a semi-restricted Boltzmann machine with a layer of visible neurons and another layer of hidden binary neurons, so it could serve as the building block for a multilayered deep-learning system. We then demonstrate that the Hopfield network can learn to form a faithful internal representation of the observed samples, with the learned memory patterns being prototypes of the input data. Furthermore, we propose a spectral method to extract a small set of concepts (idealized prototypes) as the most concise summary or abstraction of the empirical data.