Curriculum Learning for Handwritten Text Line Recognition
This addresses slow training times for researchers and practitioners in handwritten text recognition, though it is incremental as it applies an existing curriculum learning principle to a specific domain.
The paper tackles slow convergence in training Recurrent Neural Networks for handwritten text line recognition by proposing Curriculum Learning, which starts with short sequences before full lines, resulting in significantly faster training and performance improvements on three databases (Rimes, IAM, OpenHaRT).
Recurrent Neural Networks (RNN) have recently achieved the best performance in off-line Handwriting Text Recognition. At the same time, learning RNN by gradient descent leads to slow convergence, and training times are particularly long when the training database consists of full lines of text. In this paper, we propose an easy way to accelerate stochastic gradient descent in this set-up, and in the general context of learning to recognize sequences. The principle is called Curriculum Learning, or shaping. The idea is to first learn to recognize short sequences before training on all available training sequences. Experiments on three different handwritten text databases (Rimes, IAM, OpenHaRT) show that a simple implementation of this strategy can significantly speed up the training of RNN for Text Recognition, and even significantly improve performance in some cases.