LGMay 1, 2021
RotLSTM: Rotating Memories in Recurrent Neural Networks
arXiv:2105.00357v11 citations
Originality Incremental advance
AI Analysis
This is an incremental improvement for natural language processing and time series modeling, specifically targeting the bAbI dataset.
The paper tackled the problem of improving LSTM performance on time series tasks by introducing rotation matrices to modify the cell state, resulting in significant performance increases on some bAbI dataset tasks.
Long Short-Term Memory (LSTM) units have the ability to memorise and use long-term dependencies between inputs to generate predictions on time series data. We introduce the concept of modifying the cell state (memory) of LSTMs using rotation matrices parametrised by a new set of trainable weights. This addition shows significant increases of performance on some of the tasks from the bAbI dataset.