A Text Attention Network for Spatial Deformation Robust Scene Text Image Super-resolution
This work addresses the challenge of improving readability in low-resolution scene text images with deformations, which is important for applications like text recognition in real-world scenarios, though it is incremental as it builds on existing CNN-based methods.
The paper tackles the problem of reconstructing high-resolution images for spatially deformed texts (rotated and curved) in scene text image super-resolution by proposing a CNN-based Text ATTention network (TATT) that uses a transformer-based module with global attention and a text structure consistency loss. Experiments on the TextZoom dataset show TATT achieves state-of-the-art performance in PSNR/SSIM metrics and significantly improves recognition accuracy, especially for multi-orientation and curved texts.
Scene text image super-resolution aims to increase the resolution and readability of the text in low-resolution images. Though significant improvement has been achieved by deep convolutional neural networks (CNNs), it remains difficult to reconstruct high-resolution images for spatially deformed texts, especially rotated and curve-shaped ones. This is because the current CNN-based methods adopt locality-based operations, which are not effective to deal with the variation caused by deformations. In this paper, we propose a CNN based Text ATTention network (TATT) to address this problem. The semantics of the text are firstly extracted by a text recognition module as text prior information. Then we design a novel transformer-based module, which leverages global attention mechanism, to exert the semantic guidance of text prior to the text reconstruction process. In addition, we propose a text structure consistency loss to refine the visual appearance by imposing structural consistency on the reconstructions of regular and deformed texts. Experiments on the benchmark TextZoom dataset show that the proposed TATT not only achieves state-of-the-art performance in terms of PSNR/SSIM metrics, but also significantly improves the recognition accuracy in the downstream text recognition task, particularly for text instances with multi-orientation and curved shapes. Code is available at https://github.com/mjq11302010044/TATT.