A Gentle Tutorial of Recurrent Neural Network with Error Backpropagation
This is an incremental tutorial aimed at helping learners understand RNN backpropagation, with no new research contributions.
The paper provides a tutorial on recurrent neural networks (RNNs), focusing on explaining the error backpropagation step, which is challenging due to feedback loops, and details its application to long short-term memory (LSTM) units by unfolding them.
We describe recurrent neural networks (RNNs), which have attracted great attention on sequential tasks, such as handwriting recognition, speech recognition and image to text. However, compared to general feedforward neural networks, RNNs have feedback loops, which makes it a little hard to understand the backpropagation step. Thus, we focus on basics, especially the error backpropagation to compute gradients with respect to model parameters. Further, we go into detail on how error backpropagation algorithm is applied on long short-term memory (LSTM) by unfolding the memory unit.