Semantic Parsing of Interpage Relations
This work addresses the need for better semantic understanding in document digitization efforts, particularly for applications like archiving or information retrieval, but it is incremental as it builds on existing multimodal and dependency parsing approaches.
The paper tackled the problem of capturing finer semantic relations between pages in multi-page documents by formalizing it as semantic parsing of interpage relations, resulting in a method that increased LAS by 41 percentage points for semantic parsing, accuracy by 33 percentage points for page stream segmentation, and 45 percentage points for page classification over a naive baseline.
Page-level analysis of documents has been a topic of interest in digitization efforts, and multimodal approaches have been applied to both classification and page stream segmentation. In this work, we focus on capturing finer semantic relations between pages of a multi-page document. To this end, we formalize the task as semantic parsing of interpage relations and we propose an end-to-end approach for interpage dependency extraction, inspired by the dependency parsing literature. We further design a multi-task training approach to jointly optimize for page embeddings to be used in segmentation, classification, and parsing of the page dependencies using textual and visual features extracted from the pages. Moreover, we also combine the features from two modalities to obtain multimodal page embeddings. To the best of our knowledge, this is the first study to extract rich semantic interpage relations from multi-page documents. Our experimental results show that the proposed method increased LAS by 41 percentage points for semantic parsing, increased accuracy by 33 percentage points for page stream segmentation, and 45 percentage points for page classification over a naive baseline.