CREPE: Coordinate-Aware End-to-End Document Parser
This work addresses document understanding for researchers and practitioners by offering a cost-efficient, adaptable method that mitigates error propagation in OCR-dependent systems, though it appears incremental in integrating existing techniques.
The authors tackled the problem of visual document understanding by developing CREPE, an OCR-free sequence generation model that parses text and extracts spatial coordinates from document images, achieving state-of-the-art performance on parsing tasks.
In this study, we formulate an OCR-free sequence generation model for visual document understanding (VDU). Our model not only parses text from document images but also extracts the spatial coordinates of the text based on the multi-head architecture. Named as Coordinate-aware End-to-end Document Parser (CREPE), our method uniquely integrates these capabilities by introducing a special token for OCR text, and token-triggered coordinate decoding. We also proposed a weakly-supervised framework for cost-efficient training, requiring only parsing annotations without high-cost coordinate annotations. Our experimental evaluations demonstrate CREPE's state-of-the-art performances on document parsing tasks. Beyond that, CREPE's adaptability is further highlighted by its successful usage in other document understanding tasks such as layout analysis, document visual question answering, and so one. CREPE's abilities including OCR and semantic parsing not only mitigate error propagation issues in existing OCR-dependent methods, it also significantly enhance the functionality of sequence generation models, ushering in a new era for document understanding studies.