Accurate Scene Text Recognition with Efficient Model Scaling and Cloze Self-Distillation
This work improves scene text recognition for real-world applications by enhancing model efficiency and robustness to label noise, though it is incremental in nature.
The paper tackles scene text recognition by showing that scaling the decoder yields greater performance gains than encoder scaling and addresses label noise with Cloze Self-Distillation, achieving state-of-the-art results on 10 out of 11 benchmarks with reduced parameters and computational costs.
Scaling architectures have been proven effective for improving Scene Text Recognition (STR), but the individual contribution of vision encoder and text decoder scaling remain under-explored. In this work, we present an in-depth empirical analysis and demonstrate that, contrary to previous observations, scaling the decoder yields significant performance gains, always exceeding those achieved by encoder scaling alone. We also identify label noise as a key challenge in STR, particularly in real-world data, which can limit the effectiveness of STR models. To address this, we propose Cloze Self-Distillation (CSD), a method that mitigates label noise by distilling a student model from context-aware soft predictions and pseudolabels generated by a teacher model. Additionally, we enhance the decoder architecture by introducing differential cross-attention for STR. Our methodology achieves state-of-the-art performance on 10 out of 11 benchmarks using only real data, while significantly reducing the parameter size and computational costs.