Pay Attention to What You Read: Non-recurrent Handwritten Text-Line Recognition
This work addresses the need for more efficient and flexible handwriting recognition systems, particularly for handling diverse writing styles and rare words, though it is incremental as it builds on existing transformer architectures.
The authors tackled the problem of handwritten text-line recognition by introducing a non-recurrent transformer-based method, achieving significant improvements in accuracy and enabling recognition of out-of-vocabulary words, with satisfactory results even in few-shot learning scenarios.
The advent of recurrent neural networks for handwriting recognition marked an important milestone reaching impressive recognition accuracies despite the great variability that we observe across different writing styles. Sequential architectures are a perfect fit to model text lines, not only because of the inherent temporal aspect of text, but also to learn probability distributions over sequences of characters and words. However, using such recurrent paradigms comes at a cost at training stage, since their sequential pipelines prevent parallelization. In this work, we introduce a non-recurrent approach to recognize handwritten text by the use of transformer models. We propose a novel method that bypasses any recurrence. By using multi-head self-attention layers both at the visual and textual stages, we are able to tackle character recognition as well as to learn language-related dependencies of the character sequences to be decoded. Our model is unconstrained to any predefined vocabulary, being able to recognize out-of-vocabulary words, i.e. words that do not appear in the training vocabulary. We significantly advance over prior art and demonstrate that satisfactory recognition accuracies are yielded even in few-shot learning scenarios.