Understanding LSTM -- a tutorial into Long Short-Term Memory Recurrent Neural Networks
This is an incremental contribution aimed at researchers and practitioners seeking clearer explanations of LSTM-RNNs.
The paper tackles the problem of understanding LSTM-RNNs by providing a tutorial that clarifies their evolution and effectiveness, resulting in improved documentation, error corrections, and unified notation.
Long Short-Term Memory Recurrent Neural Networks (LSTM-RNN) are one of the most powerful dynamic classifiers publicly known. The network itself and the related learning algorithms are reasonably well documented to get an idea how it works. This paper will shed more light into understanding how LSTM-RNNs evolved and why they work impressively well, focusing on the early, ground-breaking publications. We significantly improved documentation and fixed a number of errors and inconsistencies that accumulated in previous publications. To support understanding we as well revised and unified the notation used.