Reservoir memory machines
This addresses a training bottleneck for researchers working on neural networks with external memory, though it appears incremental as it builds on existing echo state networks.
The authors tackled the problem of Neural Turing Machines being hard to train by proposing reservoir memory machines, which solve similar benchmark tests but train much faster, requiring only an alignment algorithm and linear regression.
In recent years, Neural Turing Machines have gathered attention by joining the flexibility of neural networks with the computational capabilities of Turing machines. However, Neural Turing Machines are notoriously hard to train, which limits their applicability. We propose reservoir memory machines, which are still able to solve some of the benchmark tests for Neural Turing Machines, but are much faster to train, requiring only an alignment algorithm and linear regression. Our model can also be seen as an extension of echo state networks with an external memory, enabling arbitrarily long storage without interference.