Fine-tuning Is a Surprisingly Effective Domain Adaptation Baseline in Handwriting Recognition
This provides a strong baseline for domain adaptation in handwriting recognition, though it is incremental as it applies an existing method to a specific domain.
The paper tackled adapting handwriting recognition models to small target datasets, showing that simple fine-tuning with data augmentation is highly effective and resistant to overfitting, achieving average relative CER improvements of 25% for 16 text lines and 50% for 256 text lines.
In many machine learning tasks, a large general dataset and a small specialized dataset are available. In such situations, various domain adaptation methods can be used to adapt a general model to the target dataset. We show that in the case of neural networks trained for handwriting recognition using CTC, simple fine-tuning with data augmentation works surprisingly well in such scenarios and that it is resistant to overfitting even for very small target domain datasets. We evaluated the behavior of fine-tuning with respect to augmentation, training data size, and quality of the pre-trained network, both in writer-dependent and writer-independent settings. On a large real-world dataset, fine-tuning on new writers provided an average relative CER improvement of 25 % for 16 text lines and 50 % for 256 text lines.