Efficient LSTM Training with Eligibility Traces
This addresses computational and biological limitations in training recurrent neural networks, though it is incremental as it adapts an existing method to LSTMs.
The paper tackled the inefficiency of backpropagation through time (BPTT) for training LSTMs by applying e-prop, a biologically plausible alternative, showing it can achieve learning on long sequences and outperform BPTT on one supervised benchmark.
Training recurrent neural networks is predominantly achieved via backpropagation through time (BPTT). However, this algorithm is not an optimal solution from both a biological and computational perspective. A more efficient and biologically plausible alternative for BPTT is e-prop. We investigate the applicability of e-prop to long short-term memorys (LSTMs), for both supervised and reinforcement learning (RL) tasks. We show that e-prop is a suitable optimization algorithm for LSTMs by comparing it to BPTT on two benchmarks for supervised learning. This proves that e-prop can achieve learning even for problems with long sequences of several hundred timesteps. We introduce extensions that improve the performance of e-prop, which can partially be applied to other network architectures. With the help of these extensions we show that, under certain conditions, e-prop can outperform BPTT for one of the two benchmarks for supervised learning. Finally, we deliver a proof of concept for the integration of e-prop to RL in the domain of deep recurrent Q-learning.