Zero-Shot Synthetic-to-Real Handwritten Text Recognition via Task Analogies
This work enables zero-shot adaptation of HTR models to new languages without any real data, which is valuable for low-resource languages.
The paper addresses zero-shot synthetic-to-real generalization for handwritten text recognition, where no real data from the target language is available. The proposed method learns parameter corrections from source languages and transfers them to target languages, achieving consistent improvements over synthetic-only baselines across five languages and six architectures.
Handwritten Text Recognition (HTR) models trained on synthetic handwriting often struggle to generalize to real text, and existing adaptation methods still require real samples from the target domain. In this work, we tackle the fully zero-shot synthetic-to-real generalization setting, where no real data from the target language is available. Our approach learns how model parameters change when moving from synthetic to real handwriting in one or more source languages and transfers this learned correction to new target languages. When using multiple sources, we rely on linguistic similarity to weigh their contrubition when combining them. Experiments across five languages and six architectures show consistent improvements over synthetic-only baselines and reveal that the transferred corrections benefit even languages unrelated to the sources.