I2C2W: Image-to-Character-to-Word Transformers for Accurate Scene Text Recognition
This addresses the challenge of accurate text recognition in noisy real-world scenes for applications like document analysis and autonomous systems, representing a novel approach rather than an incremental improvement.
The paper tackles the problem of scene text recognition under geometric and photometric degradation by proposing I2C2W, which decomposes recognition into image-to-character and character-to-word tasks, outperforming state-of-the-art methods on challenging datasets with large margins and achieving competitive performance on normal datasets.
Leveraging the advances of natural language processing, most recent scene text recognizers adopt an encoder-decoder architecture where text images are first converted to representative features and then a sequence of characters via `sequential decoding'. However, scene text images suffer from rich noises of different sources such as complex background and geometric distortions which often confuse the decoder and lead to incorrect alignment of visual features at noisy decoding time steps. This paper presents I2C2W, a novel scene text recognition technique that is tolerant to geometric and photometric degradation by decomposing scene text recognition into two inter-connected tasks. The first task focuses on image-to-character (I2C) mapping which detects a set of character candidates from images based on different alignments of visual features in an non-sequential way. The second task tackles character-to-word (C2W) mapping which recognizes scene text by decoding words from the detected character candidates. The direct learning from character semantics (instead of noisy image features) corrects falsely detected character candidates effectively which improves the final text recognition accuracy greatly. Extensive experiments over nine public datasets show that the proposed I2C2W outperforms the state-of-the-art by large margins for challenging scene text datasets with various curvature and perspective distortions. It also achieves very competitive recognition performance over multiple normal scene text datasets.