Neural Turing Machines
This work addresses the limitation of neural networks in handling algorithmic tasks, offering a differentiable architecture that could enhance machine learning for sequential and memory-intensive problems.
The authors tackled the problem of extending neural networks with external memory to enable algorithmic learning, achieving preliminary success in inferring simple algorithms like copying, sorting, and associative recall from examples.
We extend the capabilities of neural networks by coupling them to external memory resources, which they can interact with by attentional processes. The combined system is analogous to a Turing Machine or Von Neumann architecture but is differentiable end-to-end, allowing it to be efficiently trained with gradient descent. Preliminary results demonstrate that Neural Turing Machines can infer simple algorithms such as copying, sorting, and associative recall from input and output examples.