PEAN: A Diffusion-Based Prior-Enhanced Attention Network for Scene Text Image Super-Resolution
This work addresses the problem of enhancing low-resolution scene text images for better downstream recognition, representing an incremental improvement in a domain-specific area.
The paper tackles scene text image super-resolution (STISR) by proposing PEAN, a network that uses attention-based modulation and diffusion-based text prior enhancement to improve resolution and readability, achieving new state-of-the-art results on the TextZoom benchmark.
Scene text image super-resolution (STISR) aims at simultaneously increasing the resolution and readability of low-resolution scene text images, thus boosting the performance of the downstream recognition task. Two factors in scene text images, visual structure and semantic information, affect the recognition performance significantly. To mitigate the effects from these factors, this paper proposes a Prior-Enhanced Attention Network (PEAN). Specifically, an attention-based modulation module is leveraged to understand scene text images by neatly perceiving the local and global dependence of images, despite the shape of the text. Meanwhile, a diffusion-based module is developed to enhance the text prior, hence offering better guidance for the SR network to generate SR images with higher semantic accuracy. Additionally, a multi-task learning paradigm is employed to optimize the network, enabling the model to generate legible SR images. As a result, PEAN establishes new SOTA results on the TextZoom benchmark. Experiments are also conducted to analyze the importance of the enhanced text prior as a means of improving the performance of the SR network. Code is available at https://github.com/jdfxzzy/PEAN.