Transformer-Based Approach for Joint Handwriting and Named Entity Recognition in Historical documents
This work addresses the challenging task of information extraction from handwritten historical documents for researchers and archivists, presenting an incremental improvement by applying transformers to this domain.
The authors tackled the problem of jointly performing handwriting transcription and named entity recognition in historical documents by proposing an end-to-end transformer-based approach, achieving new state-of-the-art performance on the ICDAR 2017 Information Extraction competition with the Esposalles database without using dictionaries, language modeling, or post-processing.
The extraction of relevant information carried out by named entities in handwriting documents is still a challenging task. Unlike traditional information extraction approaches that usually face text transcription and named entity recognition as separate subsequent tasks, we propose in this paper an end-to-end transformer-based approach to jointly perform these two tasks. The proposed approach operates at the paragraph level, which brings two main benefits. First, it allows the model to avoid unrecoverable early errors due to line segmentation. Second, it allows the model to exploit larger bi-dimensional context information to identify the semantic categories, reaching a higher final prediction accuracy. We also explore different training scenarios to show their effect on the performance and we demonstrate that a two-stage learning strategy can make the model reach a higher final prediction accuracy. As far as we know, this work presents the first approach that adopts the transformer networks for named entity recognition in handwritten documents. We achieve the new state-of-the-art performance in the ICDAR 2017 Information Extraction competition using the Esposalles database, for the complete task, even though the proposed technique does not use any dictionaries, language modeling, or post-processing.