Unsupervised Training Data Generation of Handwritten Formulas using Generative Adversarial Networks with Self-Attention
This addresses a data scarcity problem for researchers in document analysis and OCR, though it is incremental as it builds on existing GAN methods.
The paper tackles the lack of labeled training data for handwritten mathematical expression recognition by generating a large synthesized dataset of hundreds of thousands of formulas using an attention-based GAN, and shows feasibility on the CROHME 2014 benchmark.
The recognition of handwritten mathematical expressions in images and video frames is a difficult and unsolved problem yet. Deep convectional neural networks are basically a promising approach, but typically require a large amount of labeled training data. However, such a large training dataset does not exist for the task of handwritten formula recognition. In this paper, we introduce a system that creates a large set of synthesized training examples of mathematical expressions which are derived from LaTeX documents. For this purpose, we propose a novel attention-based generative adversarial network to translate rendered equations to handwritten formulas. The datasets generated by this approach contain hundreds of thousands of formulas, making it ideal for pretraining or the design of more complex models. We evaluate our synthesized dataset and the recognition approach on the CROHME 2014 benchmark dataset. Experimental results demonstrate the feasibility of the approach.