Cells in Multidimensional Recurrent Neural Networks
This work addresses a stability problem in MDRNNs for tasks like handwritten text transcription, offering an incremental improvement by substituting LSTM cells with new designs.
The paper tackled the instability of LSTM cells in multi-dimensional recurrent neural networks (MDRNNs) by defining properties for one-dimensional LSTM and extending them to multi-dimensional cases, introducing new cells with better stability that improved recognition rates on the IFN/ENIT and Rimes databases compared to LSTM cells.
The transcription of handwritten text on images is one task in machine learning and one solution to solve it is using multi-dimensional recurrent neural networks (MDRNN) with connectionist temporal classification (CTC). The RNNs can contain special units, the long short-term memory (LSTM) cells. They are able to learn long term dependencies but they get unstable when the dimension is chosen greater than one. We defined some useful and necessary properties for the one-dimensional LSTM cell and extend them in the multi-dimensional case. Thereby we introduce several new cells with better stability. We present a method to design cells using the theory of linear shift invariant systems. The new cells are compared to the LSTM cell on the IFN/ENIT and Rimes database, where we can improve the recognition rate compared to the LSTM cell. So each application where the LSTM cells in MDRNNs are used could be improved by substituting them by the new developed cells.