A GRU-based Encoder-Decoder Approach with Attention for Online Handwritten Mathematical Expression Recognition
This work addresses the problem of accurately recognizing handwritten math expressions for applications in education and document digitization, representing a strong specific gain in the field.
The paper tackles online handwritten mathematical expression recognition by proposing a GRU-based encoder-decoder model with attention, achieving a state-of-the-art expression recognition accuracy of 52.43% on the CROHME 2014 dataset.
In this study, we present a novel end-to-end approach based on the encoder-decoder framework with the attention mechanism for online handwritten mathematical expression recognition (OHMER). First, the input two-dimensional ink trajectory information of handwritten expression is encoded via the gated recurrent unit based recurrent neural network (GRU-RNN). Then the decoder is also implemented by the GRU-RNN with a coverage-based attention model. The proposed approach can simultaneously accomplish the symbol recognition and structural analysis to output a character sequence in LaTeX format. Validated on the CROHME 2014 competition task, our approach significantly outperforms the state-of-the-art with an expression recognition accuracy of 52.43% by only using the official training dataset. Furthermore, the alignments between the input trajectories of handwritten expressions and the output LaTeX sequences are visualized by the attention mechanism to show the effectiveness of the proposed method.