Offline Handwritten Mathematical Recognition using Adversarial Learning and Transformers
This addresses a challenging problem in mathematical expression recognition for applications like digitizing handwritten notes, though it appears incremental.
The paper tackles offline handwritten mathematical expression recognition by proposing an encoder-decoder model with adversarial learning and transformers, improving expression rate by approximately 4% on the CROHME 2019 test set.
Offline Handwritten Mathematical Expression Recognition (HMER) is a major area in the field of mathematical expression recognition. Offline HMER is often viewed as a much harder problem as compared to online HMER due to a lack of temporal information and variability in writing style. In this paper, we purpose a encoder-decoder model that uses paired adversarial learning. Semantic-invariant features are extracted from handwritten mathematical expression images and their printed mathematical expression counterpart in the encoder. Learning of semantic-invariant features combined with the DenseNet encoder and transformer decoder, helped us to improve the expression rate from previous studies. Evaluated on the CROHME dataset, we have been able to improve latest CROHME 2019 test set results by 4% approx.