Chargrid: Towards Understanding 2D Documents
This addresses the challenge of extracting information from structured documents like invoices, which is incremental as it builds on existing methods by incorporating 2D layout.
The authors tackled the problem of understanding structured documents by introducing a 2D grid representation of characters to preserve layout, and they demonstrated that their pipeline significantly outperforms sequential text or image-based approaches on an information extraction task from invoices.
We introduce a novel type of text representation that preserves the 2D layout of a document. This is achieved by encoding each document page as a two-dimensional grid of characters. Based on this representation, we present a generic document understanding pipeline for structured documents. This pipeline makes use of a fully convolutional encoder-decoder network that predicts a segmentation mask and bounding boxes. We demonstrate its capabilities on an information extraction task from invoices and show that it significantly outperforms approaches based on sequential text or document images.