HTR-ConvText: Leveraging Convolution and Textual Information for Handwritten Text Recognition
This work addresses the problem of recognizing diverse handwritten texts for applications in digitization and accessibility, representing an incremental improvement over existing methods.
The paper tackled the challenge of Handwritten Text Recognition by proposing HTR-ConvText, a model that integrates convolutional and textual information to capture local and global features, achieving improved performance and generalization on datasets like IAM and READ2016, especially with limited training data.
Handwritten Text Recognition remains challenging due to the limited data, high writing style variance, and scripts with complex diacritics. Existing approaches, though partially address these issues, often struggle to generalize without massive synthetic data. To address these challenges, we propose HTR-ConvText, a model designed to capture fine-grained, stroke-level local features while preserving global contextual dependencies. In the feature extraction stage, we integrate a residual Convolutional Neural Network backbone with a MobileViT with Positional Encoding block. This enables the model to both capture structural patterns and learn subtle writing details. We then introduce the ConvText encoder, a hybrid architecture combining global context and local features within a hierarchical structure that reduces sequence length for improved efficiency. Additionally, an auxiliary module injects textual context to mitigate the weakness of Connectionist Temporal Classification. Evaluations on IAM, READ2016, LAM and HANDS-VNOnDB demonstrate that our approach achieves improved performance and better generalization compared to existing methods, especially in scenarios with limited training samples and high handwriting diversity.