A Question-Answering Approach to Key Value Pair Extraction from Form-like Document Images
This addresses the challenge of robustly extracting structured information from document images for applications like data entry and digitization, representing an incremental advance in domain-specific methods.
The paper tackles the problem of extracting key-value pairs from form-like document images by proposing KVPFormer, a question-answering based approach that uses a Transformer encoder-decoder with coarse-to-fine prediction and spatial attention, achieving state-of-the-art results with F1 score improvements of 7.2% on FUNSD and 13.2% on XFUND datasets.
In this paper, we present a new question-answering (QA) based key-value pair extraction approach, called KVPFormer, to robustly extracting key-value relationships between entities from form-like document images. Specifically, KVPFormer first identifies key entities from all entities in an image with a Transformer encoder, then takes these key entities as \textbf{questions} and feeds them into a Transformer decoder to predict their corresponding \textbf{answers} (i.e., value entities) in parallel. To achieve higher answer prediction accuracy, we propose a coarse-to-fine answer prediction approach further, which first extracts multiple answer candidates for each identified question in the coarse stage and then selects the most likely one among these candidates in the fine stage. In this way, the learning difficulty of answer prediction can be effectively reduced so that the prediction accuracy can be improved. Moreover, we introduce a spatial compatibility attention bias into the self-attention/cross-attention mechanism for \Ours{} to better model the spatial interactions between entities. With these new techniques, our proposed \Ours{} achieves state-of-the-art results on FUNSD and XFUND datasets, outperforming the previous best-performing method by 7.2\% and 13.2\% in F1 score, respectively.