Learning Over Long Time Lags
This is an incremental contribution that synthesizes existing knowledge for researchers in machine learning and time-series analysis.
This paper addresses the lack of comprehensive reviews on memory models in recurrent neural networks (RNNs) by providing a fundamental review of RNNs and long short-term memory (LSTM), and surveying recent advances in memory enhancements and learning techniques for capturing long-term dependencies.
The advantage of recurrent neural networks (RNNs) in learning dependencies between time-series data has distinguished RNNs from other deep learning models. Recently, many advances are proposed in this emerging field. However, there is a lack of comprehensive review on memory models in RNNs in the literature. This paper provides a fundamental review on RNNs and long short term memory (LSTM) model. Then, provides a surveys of recent advances in different memory enhancements and learning techniques for capturing long term dependencies in RNNs.