Deep Structured Output Learning for Unconstrained Text Recognition
This addresses the problem of recognizing text in real-world images for computer vision applications, offering a novel hybrid method that advances accuracy without relying on specific lexicons.
The paper tackles unconstrained text recognition in natural images without a fixed lexicon or known word length by proposing a CNN-CRF architecture that jointly optimizes character and N-gram predictions using synthetic data. The result is improved accuracy over character-only models on standard benchmarks, achieving state-of-the-art performance in both lexicon-constrained and unconstrained scenarios.
We develop a representation suitable for the unconstrained recognition of words in natural images: the general case of no fixed lexicon and unknown length. To this end we propose a convolutional neural network (CNN) based architecture which incorporates a Conditional Random Field (CRF) graphical model, taking the whole word image as a single input. The unaries of the CRF are provided by a CNN that predicts characters at each position of the output, while higher order terms are provided by another CNN that detects the presence of N-grams. We show that this entire model (CRF, character predictor, N-gram predictor) can be jointly optimised by back-propagating the structured output loss, essentially requiring the system to perform multi-task learning, and training uses purely synthetically generated data. The resulting model is a more accurate system on standard real-world text recognition benchmarks than character prediction alone, setting a benchmark for systems that have not been trained on a particular lexicon. In addition, our model achieves state-of-the-art accuracy in lexicon-constrained scenarios, without being specifically modelled for constrained recognition. To test the generalisation of our model, we also perform experiments with random alpha-numeric strings to evaluate the method when no visual language model is applicable.