Neural Random-Access Machines
This addresses the challenge of making neural networks handle complex data structures and algorithmic operations, which is incremental as it builds on existing neural architectures with external memory.
The authors tackled the problem of enabling neural networks to perform algorithmic tasks requiring pointer manipulation and dereferencing by proposing the Neural Random Access Machine, which can manipulate pointers to external memory and was shown to learn solutions for tasks like linked-lists and binary trees, with generalization to arbitrary-length sequences for easier tasks and constant-time memory access under certain assumptions.
In this paper, we propose and investigate a new neural network architecture called Neural Random Access Machine. It can manipulate and dereference pointers to an external variable-size random-access memory. The model is trained from pure input-output examples using backpropagation. We evaluate the new model on a number of simple algorithmic tasks whose solutions require pointer manipulation and dereferencing. Our results show that the proposed model can learn to solve algorithmic tasks of such type and is capable of operating on simple data structures like linked-lists and binary trees. For easier tasks, the learned solutions generalize to sequences of arbitrary length. Moreover, memory access during inference can be done in a constant time under some assumptions.