Joint Recognition of Handwritten Text and Named Entities with a Neural End-to-end Model
This work addresses the issue of error propagation in sequential tasks for extracting information from handwritten documents, which is incremental as it applies an existing architecture to a joint task.
The authors tackled the problem of information extraction from handwritten documents by jointly performing text transcription and named entity recognition with a single neural network, eliminating the need for separate modules. Results on historical marriage records show performance comparable to state-of-the-art methods in the ICDAR 2017 competition, without using dictionaries, language modeling, or post-processing.
When extracting information from handwritten documents, text transcription and named entity recognition are usually faced as separate subsequent tasks. This has the disadvantage that errors in the first module affect heavily the performance of the second module. In this work we propose to do both tasks jointly, using a single neural network with a common architecture used for plain text recognition. Experimentally, the work has been tested on a collection of historical marriage records. Results of experiments are presented to show the effect on the performance for different configurations: different ways of encoding the information, doing or not transfer learning and processing at text line or multi-line region level. The results are comparable to state of the art reported in the ICDAR 2017 Information Extraction competition, even though the proposed technique does not use any dictionaries, language modeling or post processing.