Handwritten Text Recognition of Historical Manuscripts Using Transformer-Based Models
This work addresses the problem of digitizing historical manuscripts for cultural and scholarly access, though it is incremental as it builds on existing transformer-based models with domain-specific enhancements.
The study tackled handwritten text recognition for 16th-century Latin manuscripts by applying TrOCR with targeted preprocessing, novel data augmentations, and ensemble learning, achieving a Character Error Rate of 1.60 with a top-5 voting ensemble, representing a 42% improvement over the previous state of the art.
Historical handwritten text recognition (HTR) is essential for unlocking the cultural and scholarly value of archival documents, yet digitization is often hindered by scarce transcriptions, linguistic variation, and highly diverse handwriting styles. In this study, we apply TrOCR, a state-of-the-art transformer-based HTR model, to 16th-century Latin manuscripts authored by Rudolf Gwalther. We investigate targeted image preprocessing and a broad suite of data augmentation techniques, introducing four novel augmentation methods designed specifically for historical handwriting characteristics. We also evaluate ensemble learning approaches to leverage the complementary strengths of augmentation-trained models. On the Gwalther dataset, our best single-model augmentation (Elastic) achieves a Character Error Rate (CER) of 1.86, while a top-5 voting ensemble achieves a CER of 1.60 - representing a 50% relative improvement over the best reported TrOCR_BASE result and a 42% improvement over the previous state of the art. These results highlight the impact of domain-specific augmentations and ensemble strategies in advancing HTR performance for historical manuscripts.