Learning distant cause and effect using only local and immediate credit assignment
This addresses the challenge of enabling efficient memory in AI systems for tasks such as navigation and sequence prediction, though it is incremental as it builds on existing neural network architectures.
The paper tackles the problem of learning distant cause-and-effect relationships in sequential data by introducing a recurrent neural network memory that uses sparse coding for combinatoric encoding, achieving memory requirements one order of magnitude less than existing models like LSTM and GRU.
We present a recurrent neural network memory that uses sparse coding to create a combinatoric encoding of sequential inputs. Using several examples, we show that the network can associate distant causes and effects in a discrete stochastic process, predict partially-observable higher-order sequences, and enable a DQN agent to navigate a maze by giving it memory. The network uses only biologically-plausible, local and immediate credit assignment. Memory requirements are typically one order of magnitude less than existing LSTM, GRU and autoregressive feed-forward sequence learning models. The most significant limitation of the memory is generalization to unseen input sequences. We explore this limitation by measuring next-word prediction perplexity on the Penn Treebank dataset.