Storing non-uniformly distributed messages in networks of neural cliques
This work addresses a specific bottleneck in associative memory systems for researchers in neural networks and data storage, but appears incremental as it builds on existing sparse associative memory frameworks.
The paper tackled the problem of performance degradation in sparse associative memories when storing non-uniformly distributed messages, and introduced several strategies to enable efficient storage, analyzing and discussing these methods with a practical application example.
Associative memories are data structures that allow retrieval of stored messages from part of their content. They thus behave similarly to human brain that is capable for instance of retrieving the end of a song given its beginning. Among different families of associative memories, sparse ones are known to provide the best efficiency (ratio of the number of bits stored to that of bits used). Nevertheless, it is well known that non-uniformity of the stored messages can lead to dramatic decrease in performance. We introduce several strategies to allow efficient storage of non-uniform messages in recently introduced sparse associative memories. We analyse and discuss the methods introduced. We also present a practical application example.