Modern Methods in Associative Memory
It provides an accessible resource for researchers and practitioners to understand and apply associative memory concepts in contemporary AI, though it is incremental as a tutorial rather than presenting new research.
This tutorial introduces modern methods in associative memory, highlighting their renewed relevance due to connections with state-of-the-art AI architectures like Transformers and Diffusion Models, which enable new interpretations and design possibilities.
Associative Memories like the famous Hopfield Networks are elegant models for describing fully recurrent neural networks whose fundamental job is to store and retrieve information. In the past few years they experienced a surge of interest due to novel theoretical results pertaining to their information storage capabilities, and their relationship with SOTA AI architectures, such as Transformers and Diffusion Models. These connections open up possibilities for interpreting the computation of traditional AI networks through the theoretical lens of Associative Memories. Additionally, novel Lagrangian formulations of these networks make it possible to design powerful distributed models that learn useful representations and inform the design of novel architectures. This tutorial provides an approachable introduction to Associative Memories, emphasizing the modern language and methods used in this area of research, with practical hands-on mathematical derivations and coding notebooks.