Dropout improves Recurrent Neural Networks for Handwriting Recognition
This work addresses the challenge of overfitting in RNNs for handwriting recognition, offering a practical improvement for researchers and practitioners in the field.
The paper tackled the problem of improving recurrent neural networks (RNNs) for handwriting recognition by applying dropout regularization, resulting in greatly enhanced performance across multiple handwritten databases.
Recurrent neural networks (RNNs) with Long Short-Term memory cells currently hold the best known results in unconstrained handwriting recognition. We show that their performance can be greatly improved using dropout - a recently proposed regularization method for deep architectures. While previous works showed that dropout gave superior performance in the context of convolutional networks, it had never been applied to RNNs. In our approach, dropout is carefully used in the network so that it does not affect the recurrent connections, hence the power of RNNs in modeling sequence is preserved. Extensive experiments on a broad range of handwritten databases confirm the effectiveness of dropout on deep architectures even when the network mainly consists of recurrent and shared connections.