Maximum Likelihood Associative Memories
This work provides theoretical foundations for associative memories, which are used in applications like CPU caches and databases, but it is incremental as it focuses on deriving bounds rather than introducing a new practical method.
The paper tackles the problem of designing associative memories by applying the maximum likelihood principle to derive fundamental limits on error rates, memory requirements, and computational complexity for uniform binary data, and compares these bounds with existing Hopfield and Gripon-Berrou neural network architectures.
Associative memories are structures that store data in such a way that it can later be retrieved given only a part of its content -- a sort-of error/erasure-resilience property. They are used in applications ranging from caches and memory management in CPUs to database engines. In this work we study associative memories built on the maximum likelihood principle. We derive minimum residual error rates when the data stored comes from a uniform binary source. Second, we determine the minimum amount of memory required to store the same data. Finally, we bound the computational complexity for message retrieval. We then compare these bounds with two existing associative memory architectures: the celebrated Hopfield neural networks and a neural network architecture introduced more recently by Gripon and Berrou.